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Abstract 

This  research  explored  the  elements  required  for  the  successful 
implementation  of  the  "rolling  down-select  strategy."  Inspired  by  Lightning  Bolt 
10,  this  study  investigated  five  acquisitions  which  have  or  are  in  the  process  of 
implementing  the  “rolling  down-select  strategy.”  The  results  suggest  several 
elements  which  appear  to  contribute  to  the  successful  use  of  the  "rolling  down- 
select  strategy."  These  findings  can  be  used  by  acquisition  planners  to  aid  in 
determining  whether  or  not  the  "rolling  down-select  strategy"  is  a  viable  strategy 
for  their  particular  acquisition. 


A  QUALITATIVE  ANALYSIS  OF  THE  ELEMENTS  REQUIRED  FOR  THE 


SUCCESSFUL  IMPLEMENTATION  OF  THE  •’ROLLING  DOWN-SELECT 

STRATEGY'' 


1.  Introduction 


In  today's  post  cold  war  era,  the  military  is  facing  massive  downsizing  and 
huge  budget  cuts.  “The  defense  budget  measured  in  constant  1995  dollars,  has 
declined  from  a  peak  of  $390  billion  in  1985  to  $252  billion  in  1995-a  reduction 
of  about  35  percent  (GAO  1995a).  These  cuts  in  defense  spending  are  further 
exacerbated  by  public  perceptions  that  poor  military  management  of  the 
acquisition  process  is  wasting  scarce  tax  revenues.  As  the  GAO  reports, 
"Congress  has  long  been  concerned  that  acquisition  practices  at  federal 
agencies  are  wasteful  and  add  billions  to  acquisition  costs.  For  example,  cost 
overruns  of  more  than  100  percent  have  been  reported  in  the  Defense 
Department  (DOD)  programs”  (GAO  1995b).  The  desire  to  change  the  public’s 
attitude  towards  the  military  coupled  with  criticism  of  its  management  practices 
have  placed  the  DOD  acquisition  process  under  fire  and  subject  to  close 
scrutiny. 

According  to  GAO  report  HR  95-1 ,  the  Department  of  "Defense  is 
committed  to  reforming  its  major  weapons  systems  acquisition  process  (involving 
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about  $80  billion  a  year)  and  has  begun  to  reassess  many  of  its  most  expensive 
weapon  programs  for  opportunities  to  cutback  to  meet  anticipated  shortfalls  in 
funding  (GAO,  1995c).  The  Department  of  Defense  and  Congress  have 
instituted  many  programs  designed  to  address  the  weaknesses  in  weapons 
systems  acquisition.  In  trying  to  make  these  changes  and  improve  the  process, 
the  DOD  "has  adopted  an  acquisition  strategy  that  calls  for  proving  technologies 
before  incorporating  them  into  the  procurement  process”  (GAO,  1995c).  One 
effective  way  to  accomplish  this  task  is  by  using  what  is  known  as  the  “rolling 
down-select  strategy”  (RDS). 

Background 

The  “Rolling  Down-Select  Strategy”  (RDS)  is  an  acquisition  approach 
which  is  not  commonly  used,  but  it  has  been  used  on  major  weapons 
acquisitions.  It  first  employs  full  and  open  competition  and  results  in  awards  to 
several  contractors  for  the  first  phase  of  the  contract.  The  field  of  contractors  is 
then  selectively  narrowed  down  through  the  design,  development  and  EMD 
phases.  Throughout  the  process,  each  contractor’s  proposal  is  evaluated 
against  the  contractor’s  performance  to  decide  which  contractor  will  be  selected 
“down”  to  the  next  phase.  This  process  results  in  award  to  a  single  contractor  for 
full  scale  production.  When  this  techniques  is  used,  unique  procedures  must  be 
followed  to  maintain  and  protect  proprietary  source  selection  information,  while 
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still  allowing  the  government  to  periodically  assess  and  provide  feedback  to  the 
selected  contractors  as  they  continue  performance  through  the  acquisition 
phases  to  production. 

Problem  Statement 

One  of  the  first  decisions  acquisition  personnel  need  to  make  when 
developing  their  acquisition  management  plan  is  which  source  selection  method 
to  use  for  their  particular  acquisition.  This  is  not  an  easy  decision,  and  any 
guidance  which  could  assist  the  acquisition  personnel  in  their  decision  would  be 
helpful.  No  empirical  research  on  the  elements  required  to  implement  the 
“roiling  down-select  strategy”  has  ever  been  done.  Therefore,  acquisition 
professionals  contemplating  implementing  RDS  have  very  little  guidance  on 
when  to  implement  this  strategy.  This  thesis  hopes  to  provide  guidance  which 
acquisition  planners  can  use  to  help  decide  if  their  program  should  implement 
the  “rolling  down-select  strategy.”  To  do  this,  this  study  will  examine  the 
elements  required  for  the  successful  implementation  of  the  “rolling  down-select 
strategy.” 

This  research  was  conducted  using  a  case-study  methodology. 
Information  was  obtained  through  open-ended  interviews  conducted  with 
personnel  from  each  of  the  case  study  subjects.  Results  of  this  exploratory 
research  will  provide  insight  to  decision-makers  on  deciding  whether  or  not  to 
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use  the  “rolling  down-select  strategy”  based  on  the  elements  of  their  program. 
Further,  this  study’s  results  will  provide  a  foundation  for  further  research  into  the 
use  and  implementation  of  the  “Rolling  Down-Select-Strategy. 

Investigative  Question 

One  investigative  question  and  four  propositions  guided  this  exploratory 
case  study  research.  The  question  and  the  elements  that  made  up  the 
propositions  are  described  below.  The  question  was,  “what  are  the 
characteristics  of  an  acquisition  program  that  make  it  a  good  candidate  for 
applying  the  “Rolling  Down-Select  Strategy?” 

Below,  I  suggest  several  characteristics  which  appear  to  be  necessary  for 
the  successful  application  of  this  strategy.  The  first  element  is  that  a  program 
must  be  a  large,  state-of-the-art  development  program.  The  program  must  be  on 
a  scale  to  warrant  the  use  of  the  “rolling  down-select  strategy.”  For  simple 
programs  it  would  be  more  efficient  to  use  traditional  “total  program”  acquisition. 

A  technology  must  be  so  advanced  that  it  would  be  too  risky  to  trust  just  one 
contractor  to  develop  and  produce  it  on  a  single  award.  Once  this  proposition  is 
resolved,  acquisition  personnel  will  be  able  to  determine  if  their  program  should 
use  RDS  based  on  the  level  of  technology  as  a  criterion. 

Next,  it  appears  a  clear  separation  between  phases  is  required  in  order  for 
RDS  to  work.  Consequently,  the  acquisition  must  be  capable  of  being  broken  up 
into  distinct  phases  where  contractors  can  be  down-selected  from  one  phase  to 
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the  next.  A  clear  separation  between  phases  makes  it  possible  for  the 
government  to  evaluate  each  contractor  based  on  each  phase  alone.  There  is  a 
question  as  to  whether  or  not  this  characteristic  is  necessary:  however  the 
results  of  the  study  of  this  proposition  should  resolve  any  doubt  about  this  point. 

Since  contractor  evaluation  is  an  integral  part  of  the  process,  clear 
performance  targets  and  performance  evaluation  criteria  must  be  established 
early  in  the  process.  Specifically,  programs  which  opt  for  this  strategy  must  have 
levels  of  performance  to  be  targeted  and  evaluated.  More  objectivity,  clear 
performance  targets  and  evaluation  criteria  are  a  necessary  requirement  for  RDS 
to  function  effectively.  The  analysis  of  this  proposition  should  provide  program 
managers  with  a  better  understanding  of  how  clear  performance  targets  and 
evaluation  criteria  relate  to  the  successful  use  of  the  “rolling  down-select 
strategy.” 

Finally,  for  RDS  to  succeed,  there  must  be  sufficient  incentive  within  each 
phase  to  maintain  multiple  efforts.  The  government  must  be  receiving  some  type 
of  benefit  from  the  RDS  process  to  continue  using  it  for  successive  phases.  In 
addition,  the  contractors  must  be  compensated  enough  to  want  to  keep 
competing  for  the  next  phase.  The  results  suggest  that  if  there  are  not  enough 
funds,  or  other  tools  to  provide  incentive  to  the  contractor,  that  the  “rolling  down- 
select  strategy”  will  not  succeed. 
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Summary 


This  thesis  is  aimed  to  provide  a  clearer  understanding  of  what  elements 
are  required  to  successfully  implement  the  “rolling  down-select  strategy.”  The 
results  of  this  research  should  effectively  improve  the  acquisition  process  by 
providing  guidance  which  acquisition  personnel  can  use  when  trying  to 
determine  if  the  “rolling  down-select  strategy”  should  be  used  on  their  program. 
This  thesis  begins  with  a  review  of  the  existing  literature  and  information  on  the 
“rolling  down-select  strategy”  in  chapter  2.  The  research  design  and 
methodology  for  analysis  are  included  in  chapter  3.  Chapter  4  includes  an 
analysis  of  the  results  of  this  research.  Finally,  the  thesis  is  concluded  in  chapter 
5  where  the  conclusions  and  recommendations  for  future  research  are 
discussed. 
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2.  Literature  Review 


This  chapter  explains  the  “rolling  down-select  strategy”  as  a  means  of 
improving  the  acquisition  process.  It  begins  with  a  discussion  of  acquisition 
reform  in  general,  and  follows  with  a  description  of  the  “rolling  down-select 
strategy”  (RDS).  Next,  the  advantages  and  disadvantages  of  RDS  are 
presented  along  with  a  discussion  of  existing  knowledge  concerning  the 
conditions  for  using  the  “rolling  down-select  strategy.”  Next,  propositions 
concerning  the  elements  necessary  for  a  successful  “rolling  down-select”  are 
explored.  The  chapter  concludes  with  a  discussion  of  the  research  objectives  for 
this  thesis. 

Acquisition  Reform 

"DOD's  acquisition  reform  program  was  established  to  reduce  acquisition 
costs  while  maintaining  technological  superiority"  (GAO,  1995d).  When  an 
acquisition  program  fails,  there  is  a  major  financial  loss  to  the  Government.  The 
Government  can  no  longer  afford  to  accept  the  risks  of  such  a  loss.  A  1995 
General  Accounting  Office  report  suggests  that  one  of  the  objectives  of 
acquisition  reform  is  to  cut  down  on  the  risk  of  program  failure  (GAO,  1995c). 
Therefore,  the  Government  is  looking  for  ways  to  reduce  this  risk. 

One  of  the  primary  focuses  of  risk  reducing  acquisition  reform  is  the 
Government’s  increased  emphasis  on  performance.  Fading  are  the  days  of 
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telling  a  contractor  exactly  how  to  design  a  major  weapons  system  and  accepting 
the  risk  that  the  design  will  not  work.  We  are  increasing  our  emphasis  on 
performance  based  acquisition.  Further,  we  are  increasing  our  reliance  on 
commercial  items.  When  we  buy  a  commercial  product  off  the  shelf,  we  are 
more  confident  that  the  product  will  be  available  and  we  are  also  more  confident 
of  the  product’s  performance. 

“Rolling  Down-Select  strategy”  -  an  initiative  of  Acquisition  Reform. 

In  1992,  the  GAO  reported  that  high  risk  acquisition  strategies  were  being  used. 
These  strategies  involved  "acquisition  of  weapons  based  on  optimistic 
assumptions  about  the  maturity  and  availability  of  enabling  technologies”  (GAO, 
1995a).  Contractors  were  selected  for  acquisitions  based  on  their  proposal  and 
their  perceived  ability  to  fulfill  the  contract.  However,  these  premature  selections 
of  a  single  contractor  to  produce  a  system  before  the  technology  is  proven  tend 
to  increase  risk.  These  risks  may  be  avoided  by  adopting  alternative  acquisition 
strategies. 

Acquisition  reform  measures  have  addressed  these  alternative  acquisition 
strategies.  The  idea  is  to  keep  evaluating  our  current  system  and  try  to  find 
ways  to  make  it  more  efficient  and  effective.  In  1996,  an  Integrated  Product 
Team  created  Lightning  Bolt  #10.  This  "Lightning  Bolt  is  an  initiative  to  reduce 
the  time  from  requirements  definition  to  contract  award”  (McCarter  1997).  One 
of  the  initiatives  listed  in  Lightning  Bolt  10  was  the  “rolling  down-select  strategy.” 
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Description  of  Roiling  Down-Select  Strategies 


Lightning  Bolt  10  gives  a  thorough  definition  of  Rolling  Down-select. 

A  rolling  down-select  uses  a  structured  source  selection  for  the  initial 
award  of  multiple  contracts,  all  of  which  include  options  for  the  follow-on 
segment  of  the  program.  The  evaluation  and  selection  of  the  contractor(s) 
to  continue  in  the  follow-on  effort  occurs  during  the  performance  of  the  first 
segment.  For  example:  the  pre-Engineering  &  Manufacturing  Development 
(EMD)  phase  for  a  major  weapon  system  would  authorize  multiple 
contractors  to  proceed  with  design  (usually  completing  at  a  normal  point  in 
design  such  as  Preliminary  Design  Review)  and  include  a  priced  or 
unpriced  option  for  the  subsequent  phase.  The  government  would  use  the 
pre-EMD  phase  to  technically  evaluate  each  contractor’s  design,  relying  on 
graded  technical  and  program  reviews  as  the  basis  for  the  evaluation. 

These  reviews  would  be  structured  in  such  a  way  as  to  avoid  technical 
leveling  while  providing  the  information  necessary  to  aid  in  the  down-select. 
Near  the  end  of  the  pre-EMD  phase,  each  competitor  would  be  asked  to 
submit  a  definitive  proposal  for  the  option  effort.  The  government  would 
then  conduct  an  informal  selection  process,  exercising  a  single  option  for 
the  completion  of  EMD.  This  approach  is  applicable  to  all  competitive  multi- 
phased  acquisitions.  (“Lightning  Bolt  10,”  1997:17) 

Key  terms  used  in  the  “rolling  down-select  strategy”  from  the  NASA  supplement 

to  the  FAR  can  be  found  in  Table  1.  Generally,  the  “rolling  down-select  strategy” 

is  best  suited  for  multi-phased  acquisition.  The  term  down-select  is  defined  in 

the  National  Aeronautical  and  Space  Administration’s  supplement  to  the  FAR, 

which  states  that  down-selection  is  “the  process  of  selecting  contractors  for 

phases  subsequent  to  the  initial  phase  from  among  the  preceding  phase 

contractors”  (NASA,  1997).  A  down-select  happens  when  a  contractor  is 

selected  down  to  the  next  phase  of  the  procurement  process. 

In  her  article  in  NCMA’s  T.I.P.S.  magazine,  Barbara  D.  Connelly-Fratzke 
gives  another  definition  of  the  “rolling  down-select  strategy. 
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A  progressive  competition/down-selection  method  refers  to  a  planned  series 
of  competitions  for  a  total  program  involving  sequential  phases.  Each 
phase  has  successively  fewer  awards  among  a  group  of  competitively 
selected  participating  contractors,  finally  resulting  in  a  single  award  for  the 
development  and/or  production  of  the  system.  (Connelly-Fratzke,  1992:1) 


Table  1. 

DOD  Rolling  Down-Select  Definitions 

TERM 

DEFINITION 

Down-selection 

In  a  phased  acquisition,  the  process  of 
selecting  contractors  for  phases  subsequent  to 
the  initial  phased  from  among  the  preceding 
phase  contractors  (NASA,  1997:  4467) 

Major  System 

“That  combination  of  elements  that  will  function 
together  to  produce  the  capabilities  required  to 
fulfill  a  mission  need”  (FAR) 

Phased  Acquisition 

A  program  comprised  of  several  distinct  steps 
or  phases  where  the  realizations  of  program 
objectives  requires  a  planned,  sequential 
acquisition  of  each  step  or  phase.  The  phases 
may  be  acquired  separately,  in  combinations, 
or  through  a  down-selection  strategy  (NASA, 
1997:  4467) 

Progressive  Competition/ 
("rolling  down-select  strategy”) 

A  strategy  where  a  solicitation  is  issued  for  all 
phases  of  this  program.  The  initial  phase 
contracts  are  awarded,  and  the  contractors  for 
subsequent  phases  are  expected  to  be  chosen 
through  a  down-selection  from  among  the 
preceding  phase  contractors.  In  each  phase, 
progressively  fewer  contracts  are  awarded  until 
a  single  contractor  is  chosen  for  the  final 
phases.  Normally  all  down-selections  are 
accomplished  without  issuance  of  a  new,  formal 
solicitation.  (NASA,  1997:  4467) 

According  to  Lt.  Col.  Joe  Shearer,  one  of  the  originators  of  the  “rolling 
down-select  strategy”  on  the  Joint  Defense  Attack  Munitions  (JDAM)  program. 
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there  is  a  key  distinction  between  a  normal  down-select  and  the  rolling  down- 
select.  As  he  points  out,  the  term  ‘rolling’  applies  to  the  fact  that  in  this  process 
there  is  a  continuous  evaluation  of  the  contractor’s  performance.  During  the 
phase,  each  contractor’s  performance  is  continuously  evaluated  against  his 
proposal  and  established  selection  criteria.  By  the  time  the  phase  is  complete 
the  Government  is  able  to  roll  to  the  next  phase  by  down-selecting  the 
contractors  based  on  their  continuous  evaluations. 

This  process  was  used  on  the  JDAM  program.  This  program  was  set  up 
using  three  Government  teams  as  shown  in  Figure  1 .  One  team  was 
responsible  for  working  with  McDonnell  Douglas,  another  with  Martin-Marietta, 
and  a  Core  team  was  responsible  for  evaluating  the  performance  of  both 
contractors  along  the  way  (Shearer). 


Figure  1.  JDAM  Evaluation  Structure 
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Theoretically,  one  can  select  five  or  six  contractors  for  concept 
exploration.  Then  maybe  three  or  four  contractors  \will  be  down-selected  for  the 
program  definition  and  risk  reduction  phase.  Then  maybe  two  or  three 
contractors  will  be  down-selected  for  the  engineering  and  manufacturing 
development  phase.  Finally,  one  contractor  makes  the  final  down-selection  and 
enters  production,  fielding/deployment,  &  operational  support.  The  key  being 
that  during  each  of  these  phases  the  contractors  are  being  evaluated  on  their 
actual  performance. 

A  memorandum  from  General  Timothy  P.  Malishenko,  presents  an 
example  where  the  “rolling  down-select  strategy”  is  used  for  the  evolved 
expendable  launch  vehicle  (EELV): 

EELV  is  employing  a  down-selection  strategy  whereby  four  contractors 
are  performing  the  first  program  module  (Low  Cost  Concept  Validation- 
LCCV),  two  contractors  will  perform  the  second  program  module  (Pre- 
Engineering  and  Manufacturing  Development-Pre-EMD),  and  one 
contractor  will  perform  the  third  program  module  (EMD).  (Malishenko, 
1996) 

Lt.  Col.  Shearer  points  out  that  for  the  JDAM  project  there  were  two 
phases  of  EMD,  an  “efficiency  phase”,  and  a  “testing  phase”  and  one  phase  of 
production  as  can  be  seen  in  Figure  2.  In  the  first  phase  of  EMD.  the  contractors 
were  required  to  improve  their  product  and  make  it  more  affordable.  The  most 
promising  proposals  were  selected  to  enter  the  first  phase.  Each  contractor’s 
performance  was  evaluated  against  his  proposal  and  preset  selection  criteria 
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continuously  during  the  phase.  The  source  selection  began  with  seven 
contractors  presenting  their  proposals.  McDonnell  Douglas  and  Martin  Marietta 
were  chosen  to  compete  in  the  first  phase.  Based  on  the  continuous  evaluation 
of  its  performance  during  the  first  phase  and  the  fact  that  it  was  able  to  make  its 
system  more  affordable,  McDonnell  Douglas  was  down-selected  to  take  the 
system  to  testing  and  production. 


ktor  X 

ktor  y 

McDonnell  Douglas 

ktor  z  PHASE  I 

18M0S 

Testing  Production 

ktor  h 

V 

Martin  Marietta 

ktor  p _ 

ktor  q _ 

^ _ _ _ _ _ 

Figure  2.  JDAM  Rolling  Down-Select  Process 

The  Solicitation  Process  for  the  “rolling  down-select  strategy” 

To  better  understand  the  “rolling  down-select  strategy”  it  is  helpful  to  see 
the  solicitation  process  involved  in  a  rolling  down-select.  Connelly-Fratzke 
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describes  the  process  in  detail.  She  describes  the  “rolling  down-select  strategy” 
as  a  “planned  sequence  of  competitions  for  successive  phases  of  a  total 
program”  (Connelly-Fratzke,  1 992).  For  the  initial  phase,  what  normally 
happens  is  that  there  will  be  full  and  open  competition  allowing  all  prospective 
contractors  to  compete  for  award  for  the  initial  concept  exploration  phase.  To 
implement  the  first  phase,  the  Government  submits  an  initial  solicitation  notice 
specifying  that  the  “rolling  down-select  strategy”  phased  approach  will  be  used. 
Included  in  the  solicitation  are  the  criteria  that  will  be  used  for  evaluation  for  all 
the  phases.  This  information  is  also  put  in  the  executive  summary  letter,  and 
sections  L  and  M  of  the  solicitation.  The  contractors  respond  to  this  notice  by 
presenting  evidence  that  they  are  able  to  design,  develop,  build,  and  produce  the 
system.  As  Connelly-  Fratzke  terms  this,  the  contractor  must  be  “capable  of 
seeing  the  project  through  to  completion”  (Connelly-Fratzke,  1992).  Contractors 
are  selected  by  the  Government  based  on  how  well  their  proposals,  as  well  as 
their  performance  during  the  phase,  satisfy  the  evaluation  criteria. 

For  subsequent  phases  the  Government  posts  a  synopsis  that  “identifies 
the  procurement  as  one  phase  in  a  multiple  phased  progressive 
competition/down-selection  in  which  the  Government  only  expects  contractors 
previously  involved  to  be  likely  to  successfully  compete  for  award  of  the  next 
phase”  (Connelly-Fratzke,  1992)  However,  a  contractor  who  was  not  in  prior 
phases  may  enter  the  solicitation  process  at  a  later  phase.  A  “new”  contractor 
that  enters  the  competition  for  a  later  phase  is  provided  with  “data  and 
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information  that  exactly  replicates  the  solicitation  provided  to  the  previously 
involved  competitors”  and  any  other  information  furnished  by  the  Government  to 
ail  involved  competitors  (Connelly-Fratzke,  1992).  For  the  subsequent  phases, 
all  offerors  must  “demonstrate  an  acceptable  level  of  ‘system  maturity;’  or 
‘system  equivalency’  in  comparison  to  the  previous  phase  contractors” 
(Connelly-Fratzke,  1992).  This  demonstration  of  maturity  is  included  in  the 
contract  for  the  initial  phase.  For  contractors  entering  at  later  phases,  they  are 
given  these  requirements,  along  with  reasonable  time  to  fulfill  them.  During  the 
acquisition  phases,  the  synopsis  for  the  next  phases  “should  be  released  early 
enough  to  permit  new  potential  offerors  to  plan  for  the  system  maturity 
demonstration”  and  it  should  include  detailed  descriptions/specifications,  “and 
the  approximate  time-frame  within  which  new  potential  offerors  will  be  offered  the 
opportunity  to  demonstrate  their  proposed  systems’  maturity”  (Connelly-Fratzke, 
1992).  Formal  source  selection  procedures  outlined  in  FAR  Subpart  15.6  are 
used  to  decide  which  contractors  will  participate  in  the  next  phase  (Connelly- 
Fratzke,  1992).  In  this  manner,  contracts  are  awarded  to  fewer  contractors  for 
each  subsequent  phase  until  one  contractor  is  awarded  the  contract  for  full-scale 
production  of  the  system.  Table  2  contains  a  synopsis  of  the  unique 
characteristics  of  the  “rolling  down-select  strategy”  for  major  systems,  developed 
by  General  Timothy  Malishenko,  it  is  an  addendum  to  his  24  May,  1996,  general 
memorandum  concerning  the  use  of  the  “rolling  down-select  strategy.” 


15 


Table  2.  Unique  Characteristics  of  the  "Rolling  Down-Select  Strategy" 


Definition:  A  planned  series  of  competitions  for  a  total  program  involving  sequential  phases 

-  The  notice  of  contract  action  synopsis,  the  initial  phase  acquisition  RFP  and  the  initial  contract 
must  explain  Government's  intent  to  conduct  progressive  competition/down-selection  for  the 
"total”  system,  i.e.,  sequential  competitions  among  those  who  successfully  demonstrate  system 
performance  requirements  of  preceding  phases 

--  Proposal  prep  instructions  must  address  total  system  acquisition,  for  all  phases,  in  general 
terms 

--  Evaluation  factors  for  award  must  include  evaluation  criteria  for  the  total  system 

~  At  each  phase.  Government  reevaluates  remaining  contractors'  abilities  to  proceed  to  next 

phase 

--  Ultimately,  one  contractor  is  chosen  to  "produce"  the  system 

-  At  the  beginning  of  any  phase,  all  interested  contractors  are  given  chance  to  demonstrate  their 
ability  to  perform  and  thus  be  considered  for  the  next  phase  award,  but  it  is  "improbable"  new 
potential  offerors  can  successfully  complete 

-  Each  subsequent  synopsis  of  proposed  contract  action  and  solicitations  describes  the 
methodology  being  used  in  the  multiple-phased  progressive  competition/down-selection  and  that 
the  government  only  expects  contractors  from  previous  phase  to  be  able  to  successfully  compete 

-  Any  new  potential  offerors  are  given  copies  of  the  initial  synopsis,  solicitation,  proposal 
preparation  instructions,  evaluation  criteria,  etc.,  but  no  new  "formal"  solicitation  (RFP)  is  required 
for  second  and  subsequent  phases 

-  "Proposals"  for  subsequent  phases  can  be  deliverables  from  a  previous  phases  .  However, 
simplified  letter  RFPs  may  also  be  issued  for  subsequent  phases,  along  with  a  "model"  contract 

-  Evaluation  and  selection  of  sources  for  subsequent  phases  is  accomplished  using  formal 
source  selection  procedures 

(Malishenko,  1996) _ 


Advantages  of  “Rolling  Down-Select  Strategy” 

According  to  Mike  Zsak,  a  representative  from  the  Office  of  the  Secretary 
of  Defense  for  Acquisition  and  Technology,  "the  DOD  acquisition  process 
provides  a  framework  for  the  management  of  programs  consisting  of  a  series  of 
phases  that  are  designed  to  reduce  risk,  ensure  affordability,  and  provide 
adequate  information  for  decision  making"  (Zsak,  1996).  Rolling  down-selects 
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improve  on  this  idea  and  offer  many  possible  advantages  as  well  as  possible 
disadvantages  to  the  acquisition  process. 

One  of  the  stated  advantages  of  the  “rolling  down-select  strategy”  is  that  it 
speeds  up  the  acquisition  process.  In  the  past,  when  there  was  a  need  for  a 
down  select,  a  planned  series  of  competitions  was  held.  The  early  interpretation 
of  the  competition  in  contracting  act  "required  unrestricted  solicitation  after  each 
phase"  in  this  process  (  Malishenko,  1996).  This  re-opening  of  competition 
between  each  phase  increased  acquisition  times  "up  to  four  additional  years  for 
five  phase  programs."  For  this  reason,  the  Department  of  Defense  introduced 
the  RDS.  As  General  Malishenko  states,  this  strategy  "eliminated  gaps  between 
phases,  eliminated  [the]  requirement  for  new,  formal  solicitation  for  each  phase, 
shortened  the  acquisition  life  cycle,"  and  it  "retained  [the]  tenet  of  full  and  open 
competition”  (complying  with  CICA  requirements).  In  this  action  item.  General 
Malishenko  set  the  precedent  for  the  removal  of  the  requirement  for  justification 
and  approval  in  the  Evolved  Expendable  Launch  Vehicle  (EELV)  program 
(Malishenko  1996).  Because  there  is  no  longer  a  requirement  for  Justification 
and  Approval,  the  rolling  down-select  can  roll  from  phase  to  phase  with  smaller 
gaps  between  phases.  Figure  3  shows  how  the  “rolling  down-select  strategy” 
saves  time.  This  figure  shows  a  down-select  without  using  the  rolling  strategy 
and  a  down-select  using  the  strategy.  The  normal  down-select  strategy,  there  is 
a  large  gap  at  the  end  of  each  phase,  during  which  proposals  are  evaluated  and 
contractors  are  chosen.  For  the  “rolling  down-select  strategy”  there  is  a  smaller 
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break  in  between  phases  because  the  contractors  are  evaluated  on  their 
performance  during  each  phase,  as  well  as  on  their  final  proposals  at  the  end  of 
each  phase.  By  the  time  the  next  phase  comes  around  the  Government  already 
knows  who  it  will  be  down  selecting  based  on  its  continuous  evaluation  of  all 
contractors.  There  is  no  longer  a  need  for  a  time  demanding  separate  source 
selection  or  long  evaluation  period  between  phases.  Because  of  this  decreased 
gap,  there  is  a  time  savings. 


Figure  3.  Time  Savings  of  the  "Rolling  Down-Select  Strategy" 
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The  “rolling  down-select  strategy”  can  also  reduce  the  contractor’s 
performance  risk.  According  to  AFFAR  appendix  AA,  "the  objective  of  the  major 
source  selection  process  is  to  select  the  source  whose  proposal  has  the  highest 
degree  of  credibility  and  whose  performance  can  be  expected  to  best  meet  the 
Government’s  requirements  at  an  affordable  cost  (price)"  (AFFAR  102).  The 
rolling  down-select  process,  decreases  the  risk  of  awarding  a  complete  contract 
to  a  contractor  who  is  unable  to  perform.  In  a  process  where  an  award  is  made 
to  a  single  contractor  the  government  is  often  constrained  and  without  options.  If 
the  contractor  is  unable  to  perform  the  project,  there  are  few  choices  besides 
terminating  the  program  and  beginning  a  new  procurement.  However,  using  the 
“rolling  down-select  strategy,”  the  contractor’s  performance  is  evaluated  during 
the  phase,  and  if  a  contractor  is  unable  to  perform  satisfactorily,  he  will  not  be 
down-selected  for  the  next  phase.  For  instance,  if  contractor  X  is  down-selected 
to  continue  from  the  concept  exploration  phase  for  a  new  tactical  fighter,  but  then 
is  unable  to  turn  his  design  into  a  prototype  in  the  program  definition  and  risk 
reduction  phase,  the  government  can  choose  to  not  down-select  him  for  the 
engineering  and  manufacturing  development  phase  because  of  his  inability  to 
perform.  This  ability  to  exclude  the  contractor  from  the  next  phase  makes  it 
possible  for  the  government  to  eliminate  incompetent  contractors  and  keep  only 
competent  contractors.  In  this  way,  the  government  increases  the  chance  of 
selecting  a  contractor  who  can  best  perform  the  contract,  and  reduces  the  risk  of 
failure. 
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Another  advantage  of  the  “rolling  down-select  strategy”  is  that  it  reduces 
the  risk  of  relying  on  a  single  solution.  With  a  few  contractors  working  on  more 
than  just  one  solution  to  a  problem  there  is  an  increased  likelihood  of  a  more 
responsive  solution.  The  rolling  down-select  strategy  provides  the  Government 
with  more  feasible  options  to  meet  the  system  requirements.  For  instance, 
when  there  is  one  contractor  with  one  solution,  if  that  solution  is  fatally  flawed, 
the  system  cannot  be  built.  However,  when  more  solutions  are  taken  into  the 
later  stages  of  their  development,  the  Government  can  better  determine  which 
solution  is  best.  Through  the  use  of  rolling  down-select,  the  good  solutions  will 
shine  and  the  flawed  solutions  will  fail  before  the  final  award  has  been  made. 

Another  advantage  of  the  “rolling  down-select  strategy”  is  that  there  is 
improved  performance  from  continued  competition.  According  to  Lt  Col  Michael 
Heberling,  “competition  can  reduce  prices,  improve  quality,  and  minimize 
technical  risk.”  The  “rolling  down-select  strategy”  takes  competition  into  the 
phases  of  the  acquisition  process.  Because  of  this  prolonged  competition  during 
the  acquisition  cycle,  the  forces  of  the  market  improve  the  acquisition.  This  can 
lead  to  reductions  in  cost,  improved  quality,  and  decreased  risk.  For  example,  in 
an  acquisition  where  a  single  contractor  is  given  the  award  in  phase  one, 
competition  stops  upon  award.  Because  of  this,  many  contractors  charge 
exorbitant  prices  for  changes  to  the  contract  after  award  has  been  made.  In 
RDS,  contractors  are  motivated  to  keep  their  prices  down  and  produce  the 
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highest  quality  that  they  are  capable  of  producing  because  they  are  still  vying  for 
the  contract. 

Another  advantage  of  the  “rolling  down-select  strategy”  is  the  reduced  risk 
associated  with  incremental  development.  RDS  increases  the  likelihood  of  being 
able  to  take  advantage  of  technological  breakthroughs  from  the  incremental 
development  of  solutions.  Technology  is  changing  fast.  The  “rolling  down-select 
strategy  allows  for  new  technology  to  be  added  to  acquisition  in  between  phases 
with  greater  ease  than  does  traditional  continuous  acquisition.  For  instance,  in 
single  award  continuous  acquisition,  if  there  is  a  technological  breakthrough 
which  changes  the  program  requirements  after  contract  award,  changes  will 
have  to  be  made  to  the  contract.  Using  RDS,  if  there  is  a  technological 
breakthrough  after  phase  I,  only  the  requirements  for  the  contractors  going  on  to 
phase  II  will  have  to  be  changed  to  incorporate  it.  This  decreases  the  risk  of  the 
system  being  outdated  by  the  time  it  is  fielded. 

Disadvantages  of  the  “Rolling  Down-Select  Strategy” 

As  advantageous  as  the  “rolling  down-select  strategy  seems  to  be,  it  is 
not  without  its  disadvantages.  These  disadvantages  mainly  point  to  the  cost  of 
RDS.  One  disadvantage  is  the  increased  cost  of  awarding  multiple  contracts. 
This  includes  the  cost  for  documentation,  data,  people,  resources,  and 
overhead.  The  administration  costs  for  a  normal  acquisition  are  multiplied  by  the 
number  of  contractors  working  on  the  phase.  For  instance,  the  more  contracts 
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you  have,  the  more  contractors  you  have  to  monitor  and  evaluate.  Monitoring 
and  evaluating  multiple  contractors  requires  multiple  resources  time  and  money. 
This  makes  the  “the  rolling  down-select  strategy”  more  expensive.  Figure  4 
illustrates  the  increased  costs  involved  with  the  “rolling  down-select  strategy” 


Figure  4.  The  Cost  of  the  "Rolling  Down-Select  Strategy" 

Another  disadvantage  of  the  “rolling  down-select  strategy”  is  that  in 
today’s  downsizing  of  the  Department  of  Defense,  multiple  monitoring  may  not 
be  possible.  System  program  offices  are  expected  to  do  “more  with  less.”  so 
there  are  very  few  idle  personnel  in  SPOs  who  can  be  used  to  monitor  the 
additional  contractors.  As  was  the  case  on  the  JDAM  effort,  there  was  a  team  of 
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government  representatives  required  for  each  contractor  on  the  job,  in  addition 
to  an  overall  core  team.  The  jobs  of  each  government  team  was  to  interface  with 
the  assigned  contractor  on  a  daily  basis.  Because  of  this  daily  interaction,  the 
government  teams  were  able  to  be  responsible  for  only  one  contractor  apiece. 
This  aspect  of  RDS  requires  SPOs  to  be  larger,  so  that  there  will  be  enough 
people  to  work  with  each  of  the  contractors.  These  multiple  monitoring  efforts 
are  a  disadvantage  because  they  are  increasingly  difficult  to  accomplish  under 
downsizing  efforts. 

Another  disadvantage  of  the  “rolling  down-select  strategy”  is  that  there  is 
less  incentive  for  contractors  to  stay  on.  As  Barbara  Connelly-Fratzke  points  out, 
during  each  of  the  acquisition  phases,  contractors  are  reimbursed  for  their  costs, 
but  they  do  not  receive  large  profits.  It  is  not  until  the  final  production  award  that 
the  contractors  will  be  given  the  opportunity  to  make  any  substantial  profit  (1992 
:  10).  For  contractors  who  have  established  markets  in  the  civilian  sector,  there 
may  be  little  incentive  to  participate  in  the  down-select  process  if  it  is  possible  to 
get  higher  return  in  the  commercial  marketplace.  In  a  single  award  acquisition, 
contractors  compete  with  little  to  no  return  except  for  the  one  who  is  awarded  the 
contract.  However,  the  competition  period  lasts  only  until  one  contractor  is 
chosen  to  receive  the  award.  Through  the  “rolling  down-select”  strategy,  the 
competition  can  last  a  few  phases  before  a  single  contractor  gets  the  award. 

The  contractors  only  reap  return  for  their  efforts  if  they  are  awarded  the  final 
contract  for  full-scale  production.  At  the  same  time,  production  numbers  often 


23 


decrease  on  defense  acquisitions,  as  was  the  case  for  the  B-2,  C-17  and  C-5.  In 
light  of  programs  such  as  these  getting  canceled,  lowering  their  production  lots, 
or  never  making  it  to  full-scale  production,  production  dollars  seem  harder  to  get. 
This  dimming  light  at  the  end  of  the  tunnel  may  cause  some  contractors  not  to 
compete  on  RDS  contracts  because  of  the  protracted  time  and  effort  with  little 
return. 


Conditions  for  Use  of  “Rolling  Down-Select  Strategy” 


Part  15  of  the  FAR  (draft  dated  1997)  describes  the  Multiphase  acquisition 
technique  as  follows: 

(a)  General.  Multiphase  source  selection  may  be  appropriate  when  the 
submission  of  full  proposals  at  the  beginning  of  a  source  selection  would 
be  burdensome  for  offerors  to  prepare  and  for  Government  personnel  to 
evaluate.  Using  multiphase  techniques,  agencies  may  seek  limited 
information  initially,  make  one  or  more  down-selects,  and  request  full 
proposals  from  a  limited  number  of  offerors. 

(b)  First  phase  notice.  In  the  first  phase,  the  Government  shall  publish  a 
notice  (see  5.205)  that  provides  a  general  description  of  the  scope  or 
purpose  of  the  acquisition,  identifies  the  criteria  that  will  be  used  to 
make  the  initial  down-select  decision,  and  solicits  responses. 
Alternatively,  the  Government  may  issue  a  solicitation  that  provides  a 
more  specific  description  of  the  supplies  or  services  to  be  procured. 

The  notice  or  solicitation  may  also  inform  offerors  of  the  evaluation 
criteria  or  process  that  will  be  used  in  subsequent  down-select 
decisions.  The  notice  or  solicitation  shall  contain  sufficient  information 
to  allow  potential  offerors  to  make  an  informed  decision  about  whether 
to  participate  in  the  acquisition.  The  notice  or  solicitation  shall  advise 
offerors  that  failure  to  participate  in  the  first  phase  will  make  them 
ineligible  to  participate  in  subsequent  phases. 

(c)  First  phase  responses.  Offerors  shall  submit  the  information  requested 
in  the  notice  or  solicitation  described  in  paragraph  (b)  of  this  section. 
Information  sought  in  the  first  phase  may  be  limited  to  a  statement  of 
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qualifications  and  other  appropriate  information  (e.g.,  proposed  technical 
concept,  past  performance  information,  limited  pricing  information). 

(d)  First  phase  evaluation  and  down-select.  The  Government  shall  evaluate 
all  offerors’  submissions  in  accordance  with  the  criteria  in  the  notice  or 
solicitation  and  make  either  a  mandatory  or  advisory  down-select 
decision. 

(1)  The  Government  may  make  a  “mandatory”  down-select  if  it  identified  the 
criteria  or  process  that  will  be  used  to  evaluate  offers  in  all  phases  and 
requested  sufficient  information  (including  cost  information)  for  there  to 
be  binding  offers.  A  mandatory  down-select  allows  the  Government  to 
prohibit  offerors  from  participating  in  subsequent  phases  based  on  the 
evaluation  criteria  set  forth  in  the  notice  solicitation. 

(2)  If  the  Government  did  not  request  sufficient  information  for  there  to  be 
binding  offers  that  the  Government  could  accept  without  further 
submissions,  the  Government  must  make  an  “advisory”  down-select,  the 
Government  shall- 

(i)  Request  selected  offerors  provide  a  proposal  for  the  next  phase  of  the 
acquisition; 

(ii)  Inform  offerors  not  selected  that,  based  on  the  offeror’s  initial 
submission,  they  are  unlikely  to  receive  an  award  and  provide  them 
supporting  rationale.  Such  offerors  may,  at  their  option,  submit  a 
proposal  for  the  second  phase  which  the  Government  must  evaluate; 
and 

(iii)  Debrief  offerors  as  required  by  15.805  and  15.806  only  when  they  have 
been  formally  excluded  from  the  competition.  Advisory  down-selects  do 
not  constitute  such  exclusion. 

(e)  Subsequent  phases.  Additional  information  shall  be  sought  in  the 
second  phase  so  that  a  mandatory  down-select  or  competitive  range 
determination  can  be  performed  or  an  award  made  without  discussion, 
if  the  criteria  to  be  used  in  making  decisions  in  the  second  phase  were 
not  stated  in  the  original  notice  or  the  solicitation,  they  shall  be  identified 
to  all  remaining  offerors  at  the  start  of  this  phase.  If  desired,  the 
Government  may  conduct  additional  phases. 

(FAR  15.103). 

0MB  Circular  A-1 09  requires  using  a  phased  approach  when  acquiring  major 
systems  (0MB,  1976).  The  focus  of  this  study’s  analysis  of  the  “rolling  down- 
select  strategy”  is  based  on  major  weapon  systems.  The  FAR  defines  major 
weapon  systems  as  systems  where 
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a)  Department  of  Defense  is  responsible  for  the  system  and  the  total 
expenditures  for  research,  development,  test,  and  evaluation  for  the 
system  are  estimated  to  be  more  than  $75,000,000(based  on  fiscal 
year  1 980  constant  dollars)  or  the  eventual  total  expenditure  for  the 
acquisition  exceeds  $300,000,000(based  on  fiscal  year  1980  dollars) 

b)  A  civilian  agency  is  responsible  for  the  system  and  total  expenditures 
for  the  system  are  estimated  to  exceed  $750,000  (based  on  fiscal  year 
1980  constant  dollars)  or  the  dollar  threshold  for  a  “major  system” 
established  by  the  agency  pursuant  to  Office  of  Management  and 
Budget  Circular  A-1 09,  entitled  “Major  System  Acquisitions,”  \whichever 
is  greater;  or 

c)  The  system  is  designated  a  “major  system”  by  the  head  of  the  agency 
responsible  for  the  system. 

However,  there  are  many  types  of  weapons  systems  ranging  from  fighter 
aircraft  to  reconnaissance  satellites.  The  “rolling  down-select  strategy”  has  been 
used  on  aircraft  like  the  Spaced  Based  InfraRed  Systems  (SBIRS),  Evolved 
Expendable  Launch  Vehicle  (EELV),  Embedded  Global  Positioning 
System/Inertial  Navigation  System  (EGI),  the  Joint  Direct  Attack  Munitions 
(JDAM),  and  other  major  weapons  systems.  It  has  yet  to  be  documented  what 
program  characteristics  lend  themselves  to  the  “rolling  down-select  strategy.” 


Elements  for  a  successful  “Rolling  Down-Select  Strategy” 

To  this  point,  there  has  been  no  empirical  research  establishing  or 
identifying  the  elements  necessary  for  a  successful  “rolling  down-select 
strategy.”  In  this  study,  I  will  examine  several  elements  which  appear  to  increase 
the  likelihood  of  a  successful  rolling  down-select.  The  first  element  is  that  a 
program  must  be  a  large,  state-of-the-art  development  program.  The  Defense 
Manufacturing  Management  Guide  for  Program  Managers  defines  state  of  the  art 
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as  “  a  material  or  process  [that]  has  had  some  factory  usage,  but  was  recently 
developed  and  is  available  from  only  one  or  a  limited  number  of  sources.  These 
types  of  processes  often  provide  the  potential  for  cost  or  time  savings”  (1989:  6- 
4).  The  program  must  be  on  a  scale  to  warrant  the  use  of  the  “rolling  down- 
select  strategy.”  For  simple  programs  it  would  be  more  efficient  to  use 
traditional  “total  program”  acquisition.  When  dealing  with  state-of-the-art 
programs,  more  contractors  bring  more  possibilities  for  insight,  and  increase  the 
chances  of  success.  A  technology  must  be  so  advanced  that  it  would  be  too 
risky  for  one  contractor  to  develop  and  produce  it  on  a  single  award. 

The  next  element  required  to  make  the  “rolling  down-select  strategy” 
work  is  that  there  must  be  a  clear  separation  between  phases.  Although  one  of 
the  advantages  of  RDS  is  time  savings,  there  must  still  be  a  clear  distinction 
between  phases.  In  this  way,  contractors  will  know  when  to  stop  and  start  work, 
and  they  will  know  what  to  focus  on  without  overlap.  In  addition,  for  RDS  to  be 
set  up,  the  acquisition  must  be  capable  of  being  broken  up  into  distinct  phases 
where  contractors  can  be  down-selected  from  one  phase  to  the  next.  A  clear 
separation  between  phases  makes  it  possible  for  the  government  to  evaluate 
each  contractor  based  on  each  phase  alone. 

Next,  it  appears  that  success  in  RDS  requires  clear  performance  targets 
and  performance  evaluation  criteria.  Each  contractor  must  know  exactly  what  he 
must  do  to  be  down-selected  to  the  next  phase.  The  Government  must  spell  out 
what  they  are  looking  for  and  how  they  will  evaluate  each  contractor.  The 
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acquisition  must  lend  itself  to  the  establishment  of  levels  of  performance  to  be 
targeted  and  evaluated.  This  will  deter  protests  and  make  the  acquisition  run 
more  smoothly. 

The  next  element  is  that  there  must  be  sufficient  incentive  within  each 
phase  to  maintain  multiple  efforts.  The  government  must  be  receiving  some  type 
of  benefit  from  the  RDS  process  to  continue  using  it  for  successive  phases.  In 
addition,  the  contractors  must  be  compensated  enough  to  want  to  keep 
competing  for  the  next  phase.  These  incentives  are  necessary  to  keep  both 
players  in  the  game.  If  either  party  becomes  disinterested,  then  the  benefits  of 
the  “rolling  down-select  strategy”  will  decrease. 

Research  Objectives 

This  literature  review  presented  the  existing  literature  on  down-selects. 
According  to  government  contracting  representatives,  such  as  SMC’s  Linda 
Barnard,  there  is  no  definitive  guidance  on  the  “rolling  down-select  strategy.” 
There  is  a  need  for  definitive  guidance  which  contracting  officers  can  use  to 
decide  if  their  acquisition  should  employ  the  “rolling  down-select  strategy.”  To 
date,  the  literature  does  not  provide  guidance  for  determining  when  a  rolling 
down-select  would  be  best  implemented.  This  exploratory  study  will  try  to  begin 
to  answer  one  very  important  question:  Under  what  conditions  can  the  “rolling 
down-select  strategy”  be  used?  In  other  words,  what  elements  are  required  to 
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make  it  work?  This  researcher  expects  that  the  programs  studied  will  contain 
all  of  the  elements  contained  in  Table  3. 

Table  3.  Propositions  for  this  Research 


Proposition  1 :  RDS  can  only  be  effectively  applied  to  programs  that  push  the  state-of-the-art 


Proposition  2:  RDS  can  only  be  effectively  applied  to  programs  that  have  the  capability  for  clear 
separations  between  phases 


Proposition  3:  RDS  can  only  be  effectively  applied  to  programs  that  have  the  capability  for  clear 
performance  targets  and  performance  evaluation  criteria 


Proposition  4:  RDS  can  only  be  effectively  applied  to  programs  which  have  sufficient  incentive 
within  each  phase  to  maintain  multiple  efforts. 


To  answer  this  question,  I  will  research  the  elements  required  for  a  successful  down-select. 
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3.  Methodology 

This  exploratory  research  was  a  study  of  the  elements  necessary  for  a 
successful  implementation  of  the  “rolling  down-select  strategy.”  The  need  for 
this  study  was  based  on  the  lack  of  existing  information  concerning  this  new 
strategy  and  the  need  for  more  direction  and  guidance  in  its  use.  This  thesis 
provides  guidance  on  the  appropriate  application  of  the  “rolling-down-select 
strategy,”  and  thereby  provides  guidance  to  help  assess  the  suitability  of  a  new 
program  for  the  use  of  this  strategy. 

Organization  of  Chapter 

This  chapter  covers  the  design  of  this  research.  The  design  was  first 
employed  by  Ms.  Vicki  Fry  in  her  investigation  of  commercial  business  practices. 
It  begins  with  the  rationale  for  the  use  of  a  case  study  methodology.  This 
particular  research  design  relies  heavily  on  Yin’s  components  of  a  case  study 
design  (1)  research  or  investigative  question(s)  and  their  related  propositions;  (2) 
unit  of  analysis;  (3)  logic  linking  the  data  to  the  propositions;  and  (4)  criteria  for 
interpreting  findings  (1 984:29).  The  chapter  then  goes  on  to  discuss  how  the 
quality  of  the  research  was  maintained.  Finally,  this  chapter  covers  the  case 
study  protocol  that  was  used  for  this  research. 
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Case-Study  Design 


This  research  attempted  to  ascertain  the  elements  required  for  a 
successful  rolling  down  select  process.  According  to  Robert  Yin,  a  case  study 
design  is  appropriate  for  asking  an  exploratory  “what”  question  (Yin,  1984:7). 
Since  this  study  was  attempting  to  isolate  those  program  elements  which 
contribute  the  most  to  the  successful  implementation  of  the  “rolling  down-select 
strategy,”  it  was  believed  that  case  study  was  the  most  appropriate  research 
design  (Fry,  1995). 

According  to  Cooper  and  Emory,  case  studies  emphasize  full  contextual 
analysis  of  a  limited  number  of  events  or  conditions  and  their  interrelations 
(1 991 :  142).  Case  studies  emphasize  detail,  making  them  useful  when  it  comes 
to  problem  solving  (Cooper  and  Emory,  1991).  Yin  makes  reference  to  a  case 
study  as  an  “empirical  inquiry  that  investigates  a  contemporary  phenomenon 
within  its  real  life  context;  when  the  boundaries  between  phenomenon  and 
context  are  not  clearly  evident;  and  in  which  multiple  sources  of  evidence  are 
used  (1984:  23). 

The  “rolling  down-select  strategy”  can  only  be  studied  as  it  is  used  in 
acquiring  a  system.  By  itself  it  is  only  a  strategy.  It  is  not  until  it  is  actually 
implemented  that  it  can  be  seen  in  action.  Therefore,  the  “rolling  down-select” 
strategy  can  only  be  studied  within  the  context  of  actual  acquisitions  (Fry,  1995). 
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The  design  of  this  study  was  based  on  a  modified  case-study  design 
implemented  by  Vicki  Fry  in  her  1995  thesis  on  commercial  practices.  Like  Fry’s 
thesis,  this  study  followed  the  elements  of  a  historical  research  approach  which, 
in  part,  relied  on  interviews  for  its  research  data  (1995:  60).  This  study  was  done 
using  documents  from  contract  files  and  telephone  interviews  with  Government 
personnel  working  on  the  acquisitions.  The  nature  of  the  acquisition  process  is 
such  that  direct  observation  was  not  possible  (Fry,  1995). 

Because  there  has  been  no  empirical  research  of  the  elements  required 
for  the  successful  implementation  of  the  “rolling  down-select  strategy,”  this 
research  was  by  default  exploratory.  According  to  Cooper  and  Emory 
exploratory  research  is  conducive  to  developing  hypotheses  or  questions  for 
further  research  (1991  :  140).  This  was  one  of  the  goals  of  this  research.  There 
has  been  no  empirical  evidence  presented  which  could  facilitate  formation  of  any 
hypotheses  in  this  area.  Nor  has  there  been  enough  data  to  explore  any  such 
hypotheses  in  this  area.  A  survey  approach  would  not  have  worked  for  this 
research  because  there  have  been  only  a  limited  number  of  acquisitions  which 
have  used  the  “rolling  down-select  strategy”. 

Consequently,  the  case  study  method  was  determined  to  be  the  most 
appropriate  research  approach  for  this  study.  This  method  was  used  to  answer 
a  “what”  question  -  namely,  what  are  the  elements  required  for  the  successful 
implementation  of  the  “rolling  down-select  strategy?”  Answering  this  “what” 
question  through  a  sterile  survey  or  any  other  method  would  not  have  been 
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practical.  Through  the  use  of  open-ended  interviews,  contracting  officers  were 
given  the  opportunity  to  explain  the  elements  that  they  saw  as  being  necessary 
for  their  acquisition,  as  well  as  the  reasons  they  chose  this  strategy.  Trends  in 
their  responses,  were  used  to  make  hypotheses  for  future  research  (Fry.  1995). 

Research  Design 

The  research  design  is  the  plan  for  the  types  and  sources  of  information 
to  be  selected  to  answer  the  research  question  (Cooper  and  Emory,  1991:138). 

It  is  a  structure  for  establishing  the  relationships  between  the  study’s  variables. 
Cooper  and  Emory  also  define  the  research  design  as  a  blueprint  for  the  outline 
of  all  procedures  from  the  hypothesis  to  the  analysis  of  data  (1991  : 139).  This 
section  covers  Yin’s  research  design  elements  important  to  case-study 
methodologies.  These  elements  are  (1)  research  question(s)  and  their  related 
propositions;  (2)  the  unit  (s)  of  analysis;  (3)  logic  linking  the  data  to  the 
propositions;  and  (4)  criteria  for  interpreting  the  findings  (1984:29).  Table  4 
shows  the  research  variables  related  to  these  propositions. 

Investigative  Question.  “What  are  the  characteristics  of  an  acquisition 
program  that  make  it  a  good  candidate  for  applying  the  “Rolling  Down-Select 
Strategy?” 

Proposition  1.  This  researcher  believed  that  acquisitions  must  be  state  of 
the  art  to  best  benefit  from  the  “rolling  down-select  strategy.”  A  program  must  be 
on  a  scale  to  warrant  the  use  of  the  “rolling  down-select  strategy.”  For  simple 
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programs  it  would  be  more  efficient  to  use  traditional  “total  program”  acquisition. 
When  dealing  with  state-of-the-art  programs,  more  contractors  bring  more 
possibilities  for  insight,  and  increase  the  chances  of  success.  A  technology  must 
be  so  advanced  that  it  would  be  too  risky  for  just  one  contractor  to  develop  and 
produce  it  on  a  single  award. 

Proposition  2.  This  researcher  thought  that  acquisitions  must  have  a 
clear  separation  between  phases  to  use  RDS.  Although  one  of  the  advantages 
of  RDS  is  time  savings,  there  must  still  be  a  clear  distinction  between  phases.  In 
this  way,  contractors  will  know  when  to  stop  and  start  work,  and  they  will  know 
what  to  focus  on  without  overlap.  In  addition,  for  RDS  to  be  setup,  the 
acquisition  must  be  able  to  be  broken  up  into  distinct  phases  where  contractors 
can  be  down-selected  from  one  phase  to  the  next.  A  clear  separation  between 
phases  makes  it  possible  for  the  government  to  evaluate  each  contractor  based 
on  each  phase  alone. 

Proposition  3.  This  researcher  believed  that  acquisitions  must  have  clear 
performance  targets  and  performance  evaluation  criteria  to  be  suitable  for  using 
the  “rolling  down-select  strategy”  Each  contractor  must  know  exactly  what  he 
must  do  to  be  down-selected  to  the  next  phase.  The  Government  must  spell  out 
what  it  is  looking  for  and  how  it  will  evaluate  each  contractor.  The  acquisition 
must  be  capable  of  being  evaluated  at  different  targeted  levels  of  performance. 
This  will  deter  protests  and  make  the  acquisition  run  more  smoothly. 
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Proposition  4.  This  researcher  thought  that  acquisitions  must  have 
sufficient  incentive  within  each  phase  to  maintain  multiple  efforts  in  order  to 
implement  the  “rolling  down-select  strategy.”  The  government  must  be  receiving 
some  type  of  benefit  from  the  RDS  process  to  continue  using  it  for  successive 
phases.  In  addition,  the  contractors  must  be  compensated  enough  to  want  to 
keep  competing  for  the  next  phase.  These  incentives  are  necessary  to  keep 
both  players  in  the  game.  If  either  party  becomes  disinterested,  then  the 
benefits  of  the  “rolling  down-select  strategy”  will  decrease. 


Table  4.  Research  Variables 


Variable 

Proposition 

Definition 

Tech-Level 

1 

The  level  of  technology  of  the  system. 

Phase-Separation 

2 

The  ability  to  separate  the  program  into  clear 

phases. 

Clear-Targets 

3 

The  existence  of  clear  performance  targets. 

Ktor-Incentive 

4 

The  incentive  level  contractors  were  given  to 

perform 
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Units  of  Analysis.  The  data  used  to  answer  the  investigative  question 
were  collected  from  structured  interviews  with  procurement  personnel  who 
participated  in  the  five  Air  Force  Material  Command  (AFMC)  acquisitions  which 
have  used  the  “rolling  down-select”  strategy.  The  focus  of  the  interviews  was  on 
the  conditions  present  in  the  acquisition  strategy  before  and  after  the  “rolling 
down-select  strategy.”  This  covered  the  program  from  the  initial  mission  need  to 
RFP  and  through  the  various  program  phases.  The  cases  ended  with  the  final 
award  to  the  production  and  deployment  contractor.  To  complete  this  research, 
contracting  officers  were  also  questioned  concerning  post-award  issues  (Fry, 
1995). 

The  scope  of  cases  consisted  of  major  weapons  systems  acquisitions 
under  AFMC  for  many  reasons.  First  of  all,  according  to  an  Air  Force  Material 
Command  FY  contracting  summary,  in  Fiscal  Year  1993,  AFMC  was  responsible 
for  77.5  percent  of  all  Air  Force  contracting  dollars.  (AFMC  Contracting 
Summary,  1 994: 1 2).  AFMC  was  also  chosen  because  it  was  an  accessible 
target  sample.  The  home  base  for  this  research  was  the  headquarters  for  AFMC 
located  at  Wright-Patterson  Air  Force  Base.  Because  this  research  was  done  so 
close  to  the  headquarters  of  this  major  command,  it  was  readily  accessible  for 
the  research  (Fry,  1995). 

Potential  cases  for  this  research  were  identified  through  telephone 
interviews  with  representatives  from  Space  and  Missile  Command  (SMC), 
Aeronautical  Systems  Command  (ASC),  and  Engineering  Systems  Command 
(ESC).  From  these  calls,  five  known  acquisitions  which  had  employed  the 
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“rolling  down-select  strategy”  were  identified.  Representatives  from  each  of 
these  five  acquisitions  were  contacted  and  scheduled  for  interviews.  Interviews 
for  acquisitions  involving  personnel  at  Wright-Patterson  Air  Base  were  conducted 
in  person.  Interviews  for  acquisitions  involving  personnel  located  off-site  were 
conducted  by  telephone.  A  standardized  interview  guide  was  used  in  both 
instances  (refer  to  Appendix  A). 

Links  of  Data  to  Propositions.  The  data  for  this  analysis  was  collected 
during  a  six  week  interview  period.  Each  of  the  acquisition  representatives  was 
contacted  via  telephone  and  scheduled  for  an  interview.  Then,  each 
representative  was  sent  a  copy  of  the  interview  questions  to  be  used  to  solicit 
information.  Then  interviews  were  conducted.  The  data  collected  were  linked  to 
the  propositions  through  pattern  matching.  Yin  (1994),  and  Miles  and  Huberman 
(1984),  both  suggest  the  use  of  pattern  matching  for  this  purpose.  Pattern 
matching  was  used  here  to  connect  the  data  to  the  propositions  and  aid  in 
analysis.  As  Miles  and  Huberman  point  out,  pattern  codes  are  “explanatory  or 
inferential  codes,  that  identify  an  emergent  theme,  pattern,  or  explanation  that 
the  site  suggests  to  the  analyst”  (1984:67).  According  to  Yin,  pattern  matching 
is  one  of  the  best  strategies  for  analyzing  case  studies.  He  points  out  that  this 
strategy  “compares  an  empirically  based  pattern  with  a  predicted  one  (or  with 
several  alternative  predictions)”  (Yin,  1994  :  106). 
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Pattern  coding  was  used  in  this  research  throughout  data  coding  and 
analysis  to  aid  in  the  final  analysis.  Using  the  investigative  question, 
propositions,  and  variables,  the  researcher  developed  a  preliminary  list  of  codes 
prior  to  data  collection,  as  suggested  by  Miles  and  Huberman.  Codes  were 
assigned  initially  to  the  data  from  each  interview  or  review  of  documentation.  As 
new  themes  arose  from  the  interviews  or  documents,  they  were  added  to  the  list 
of  pattern  codes  before  the  next  set  of  data  was  collected.  By  assigning  and 
revising  codes  during  the  research,  data  was  continually  analyzed  as  suggested 
by  Miles  and  Huberman(1984:  63).  Once  all  the  data  was  collected,  final  pattern 
codes  were  assigned  (Fry,  1995). 

Due  to  the  exploratory  nature  of  this  research,  three  descriptive 
techniques  were  used  to  analyze  the  data  (Miles  and  Huberman,  1984:215). 

The  first  effort  was  to  identify  and  finalize  themes  or  patterns  in  the  data.  Miles 
and  Huberman  state  that  to  identify  a  theme  or  pattern,  the  researcher  has  to 
“isolate  something  that  happens  a  number  of  times  and  (b)  that  consistently 
happens  in  a  specific  way”  (1984:  215).  The  next  step  was  to  count  the  number 
of  occurrences  of  the  codes.  Third,  the  themes  were  categorized  based  on 
similarities  and  differences  in  the  data. 

Criteria  for  Interpreting  Findings.  The  generalizability  of  the  results  of  the 
findings  of  this  research  is  limited  to  AFMC  major  weapons  systems  acquisitions. 
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Protections  of  Quality 


In  order  to  ensure  that  the  standards  of  quality  for  construct  validity, 
internal  validity,  external  validity,  and  reliability  were  upheld,  measures 
suggested  by  Robert  Yin  were  taken.  To  ensure  construct  validity,  Yin’s 
recommended  chain  of  evidence  procedure  was  used.  This  process  involved 
developing  and  maintaining  a  case  study  database  from  which  the  research 
report  can  cite  relevant  portions.  As  Yin  notes,  this  “is  to  allow  an  external 
observer — ^the  reader  of  the  case  study,  for  example — ^to  follow  the  derivation  of 
any  evidence  from  initial  research  questions  to  ultimate  case  study  conclusions” 
(1994:98). 

Internal  validity  was  upheld  through  the  use  of  pattern  matching. 

According  to  Yin,  the  use  of  pattern  matching  makes  sure  that  inferences  made 
about  collected  data  are  accurate.  This  technique  was  also  employed  to  rule  out 
alternative  explanations.  External  validity  was  taken  care  of  through  the  analysis 
of  multiple  cases.  Multiple  cases  revealed  any  existing  replication  or  absence  of 
phenomena  across  cases.  To  protect  the  quality  criterion  of  reliability,  Yin 
recommends  using  a  case  study  protocol  and  database  as  was  done  in  this 
research  (Yin,  1994). 
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Case  Study  Protocol 


Yin’s  case  study  protocol  method  was  applied  to  this  research.  Applicable 
elements  were  (1)  study  overview;  (2)  data  sources;  (3)  case  study  questions; 
and  (4)  the  case  study  database. 

Study  Overview.  Research  participants  were  mailed  a  copy  of  the 
interview  and  informed  of  its  purpose.  They  were  given  time  to  read  through  the 
interview  questions  in  plenty  of  time  for  them  to  have  all  necessary  information  at 
interview  time. 

Data  Sources.  Data  for  this  research  was  taken  from  AFMC 
representatives’  responses  to  interview  questions.  The  questions  addressed  the 
perceptions  of  all  AFMC  contracting  personnel  involved  in  the  five  acquisitions. 
FAR  2.101  defines  a  contracting  officer  as  “a  person  with  the  authority  to  enter 
into,  administer,  and/or  terminate  contracts  and  make  related  determinations  and 
findings”  (FAR).  Program  manager  refers  to  the  individual  responsible  for  all 
parts  of  the  acquisition  as  a  whole.  The  perceptions  and  decisions  of  contracting 
officers  and  program  managers  were  important  to  this  study.  Interviews  were 
conducted  with  a  mixture  of  contracting  officers  and  program  managers  from 
each  acquisition.  Contract  file  documentation  was  solicited  and  used  to  fill  in  the 
blanks  where  questions  could  not  be  answered. 

Case  Study  Questions.  The  questions  asked  during  the  interview  were 
designed  to  address  the  investigative  question  under  investigation.  As  Yin 
(1994)  recommends,  the  questions  served  to  remind  the  researcher  of  the  data 
to  be  collected.  The  questions  asked  during  the  interview  were  open-ended  and 
dynamic  to  encourage  the  interviewee  to  respond  freely.  Follow-up  questions 
were  usually  contingent  on  responses  to  initial  questions. 
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Database.  Data  collection  in  the  field  was  documented  via  notes.  Data 
points  from  interviews  and  documents  were  collected  and  put  on  note  card.  Note 
cards  from  each  source  were  coded  differently  to  maintain  linkage  with  the 
source  of  the  data.  The  note  cards  were  kept  track  of  through  the  alpha  code, 
which  referenced  the  case;  three  numbers,  which  referred  to  a  document  or 
interview  for  that  case;  and  two  numbers  referring  to  a  data  point  on  that 
document.  Data  on  each  card  was  patter  coded.  An  example  note  card  would 
be  A-012-05.  This  refers  to  data  point  number  five  on  interview  number  twelve  of 
acquisition  A. 

Summary 

The  purpose  of  this  research  was  to  explore  the  elements  which  are 
necessary  to  make  the  “rolling  down-select  strategy”  work.  This  research  began 
with  a  hypothesis  of  the  necessary  elements  and  it  was  hoped  that  the  findings 
would  support  a  firmer  hypothesis  for  the  necessary  elements  as  well  as  shed 
light  on  any  element  which  were  not  thought  of.  This  research  direction  was 
stimulated  by  the  lack  of  guidance  in  this  new  area. 

Yin’s  (1994)  suggestions  helped  to  ensure  the  quality  of  this  research. 

Five  AFMC  major  weapons  system  acquisitions  using  the  “rolling  down-select 
strategy”  were  analyzed.  Using  multiple  case  studies  and  a  broad  investigative 
question  made  possible  the  discovery  of  interesting  findings  with  external 
validity.  Pattern  matching  upheld  internal  validity  while  the  case-study  protocol 
ensured  reliability  (Fry,  1995). 
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The  results  of  this  exploratory  effort  are  presented  and  analyzed  in 
chapter  four.  These  findings  will  aid  in  future  quantitative  research  after  more 
acquisitions  using  RDS  have  been  instituted  and  completed.  The  results  will 
also  aid  acquisition  strategy  planners  in  deciding  whether  or  not  the  “rolling 
down-select  strategy”  would  be  appropriate  for  their  acquisition. 
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4.  Results  and  Analysis 

Five  acquisitions  were  analyzed  in  detail  in  this  exploratory  research  to 
determine  the  elements  a  program  needs  to  successfully  implement  the  “rolling 
down-select  strategy.”  This  chapter  begins  with  a  background  on  the 
acquisitions  used  as  cases  for  this  research.  It  then  goes  on  to  provide  a 
summary  of  the  data  that  was  collected  and  analyzed.  This  is  followed  by  an 
analysis  of  the  findings  concerning  the  investigative  question  and  the  related 
propositions  identified  in  Chapter  3. 

Overview  of  Cases 

Telephone  interviews  with  representatives  from  Space  and  Missile 
Command  (SMC),  Aeronautical  Systems  Command  (ASC),  and  Engineering 
Systems  Command  (ESC)  identified  the  five  cases  that  fit  the  scope  of  this 
study.  The  five  programs  studied  were:  the  Evolved  Expandable  Launch 
Vehicle  (EELV),  the  Embedded  GPS/INS  (EGI),  the  Joint  Direct  Attack  Munition 
(JDAM),  the  Joint  Air-To-Surface  Standoff  Missile  (JASSM),  and  the  Space 
Based  Infrared  System  (SBIRS-HI). 

All  but  one  of  the  acquisitions  studied  were  major  weapons  systems 
acquisitions  which  incorporated  the  “rolling  down-select  strategy”  to  streamline 
their  acquisition.  The  one  outlier,  the  EGI  program  was  an  acquisition  for  the 
upgrade  of  internal  navigation  systems  for  the  department  of  defense’s  aircraft 
inventory.  EGI  also  varied  from  the  other  acquisitions  in  that  the  “rolling  down- 
select  strategy”  was  employed  to  respond  to  a  protest  which  was  made  after 
their  original  source  selection. 
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It  is  interesting  to  note  that  the  programs  used  slight  variations  of  the 
“rolling  down-select  strategy.”  The  JDAM  and  JASSM  programs  used  the  “rolling 
down-select”  process  as  defined  by  Lightning  Bolt  10.  In  these  two  programs, 
single  solicitations  were  issued  at  the  beginning  of  the  programs  and  contract 
options  were  used  to  down-select  the  contractors  to  the  next  phase.  EELV,  EGI, 
and  SBIRS-HI  used  the  process  as  implemented  by  NASA  and  Barbara 
Connelly-Fratzke  in  here  1992  TIPS  article.  These  three  programs  issued  new 
solicitations  between  each  phase.  This  difference  resulted  in  a  longer  source 
selection  period  for  these  three  programs.  The  variation  in  these  programs  was 
instrumental  in  developing  the  set  of  elements  required  for  the  “rolling  down- 
select  strategy.” 

Personnel  from  all  five  program  offices  had  little  if  any  prior  experience 
with  the  “rolling  down-select  strategy.”  In  1993,  acquisition  professionals 
developing  the  acquisition  plan  for  the  Joint  Direct  Attack  Munition  presented 
their  version  of  the  “rolling  down-select  strategy”  to  the  Secretary  of  the  Air  Force 
for  Acquisition  (SAF/AQ).  The  SAF/AQ  approved  this  strategy  and  it  was 
implemented  on  the  JDAM  program.  After  seeing  how  successful  the  strategy 
worked  for  JDAM,  the  SAF/AQ  directed  the  strategy  be  used  on  the  JASSM 
program.  The  program  manager  for  JDAM  later  became  the  program  manager 
for  JASSM.  For  EELV  and  SBIRS-HI,  the  acquisition  planners  learned  of  the 
acquisition  strategy  from  NASA  and  the  SAF/AQ.  After  analyzing  their  programs, 
they  decided  to  incorporate  the  strategy  to  streamline  their  acquisition  approach. 
EGI  had  no  intention  of  using  the  “rolling  down-select  strategy”  until  their  was  a 
successful  protest.  When  the  protest  was  upheld,  EGI  had  chose  to  use  the 
“rolling  down-select  strategy”  to  continue  competition  into  the  phase  rather  than 
restart  the  source  selection  process.  With  the  exception  of  EGI,  all  of  these 
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acquisitions  chose  RDS  with  the  ideas  of  acquisition  reform,  and  streamlined 
acquisition. 

Table  5  includes  a  summary  of  the  data  collected  in  this  research.  The 
data  is  configured  to  the  study’s  pattern  codes.  These  pattern  codes  are  defined 
in  Appendix  A. 
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Table  5.  Summary  of  Data 


CODE  EELV  EGI  JDAM  JASSM  SBIRS-HI 


State-Simple  No _ No _ No _ No _ No 

State-Practice  No _ Yes _ No _ No _ Yes 

State-Art  Yes _ No _ Yes _ Yes _ No 

State-Exp  No _ No _ No _ No _ No 

Tech-Lev-nec  Yes  No  Yes  Yes  No 


EZ-Sep _ Yes _ Yes _ Yes _ Yes _ Yes 

EZ-Sep-Nec  Yes  Yes _ Yes _ Yes _ Yes 


Clr  Targ _ Yes _ Yes _ Yes _ Yes _ ^ 

CIr-Targ-Nec  Yes  Yes _ Yes _ Yes _ Yes 


Ktor-Inc  No  No _ No _ No _ No 

Ktor-Inc-Nec  No _ No _ No _ No _ No 

Ktor-comp-CP  No _ No _ Yes _ Yes _ Yes 

Ktor-Comp-FF _ Yes _ Yes _ No _ No _ No 

Ktor-Comp-Nec  Yes  Yes  Yes _ Yes _ Yes 


Ad  Comp  Nec  Yes  Yes _ Yes _ Yes  \  Yes 


Perf-Bse-Spec  Yes  Yes  Yes _ Yes  |  Yes 


Dev-Time  Yes  Unknown  Yes  Yes  Yes 

>18<24 


Poss-Cost-Sav  No  No  Yes  Yes  Yes 


Hl-Risk  Yes  T  Yes  Yes  Yes  Yes 


Lo-Fund  Yes  I  Unknown  No  Yes  No 


Min-Sched-Imp  Yes  Yes  Yes  Yes _ ^ 
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Investigative  Question 


For  this  study,  the  fundamental  research  question  was;  What  are  the 
characteristics  of  an  acquisition  program  that  make  it  a  good  candidate  for 
applying  the  “Rolling  Down-Select  Strategy?”  In  order  to  answer  this  question, 
we  must  first  look  at  initial  propositions,  and  the  elements  discovered  in  the 
research. 

Proposition  1.  This  proposition  was  based  on  the  belief  that  acquisitions 
must  be  state  of  the  art  to  best  benefit  from  the  “rolling  down-select  strategy.” 
Three  of  the  cases,  EELV,  JDAM,  and  JASSM  were  state  of  the  art.  Two  of  the 
cases,  EGI  and  SBIRS-HI,  were  state  of  the  practice.  All  but  one  of  the  cases, 
felt  that  the  high  level  of  technology  involved  in  the  system  played  a  big  role  in 
the  decision  to  use  RDS. 

All  of  the  cases  felt  that  a  requirement  for  RDS  should  be  that  the  system 
be  at  least  state  of  the  practice  and  at  most  state  of  the  art.  All  of  the  cases  in 
this  study  believed  that  it  would  be  a  waste  of  resources  to  use  RDS  on  simple 
state  systems.  The  JDAM  program  manager  mentioned  that  he  felt  that  the  use 
of  the  “rolling  down-select  strategy”  on  an  experimental  system  was  not 
advisable.  JDAM  representatives  believed  that  using  RDS  on  an  experimental 
system  would  make  it  harder  to  measure  progress  because  of  the  theoretical 
nature  of  experimental  systems. 

Proposition  2.  This  proposition  expressed  the  belief  that  acquisitions  must 
have  a  clear  separation  between  phases  to  use  RDS.  This  proposition  was 
supported  by  the  fact  that  all  five  programs  agreed  with  the  idea  that  a  clear 
phase  separation  was  critical.  A  representative  from  EELV  said  that  clear 
separation  of  phases  was  necessary  to  aid  in  evaluating  key  cost,  schedule  and 
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performance  parameters.  SBIRS-HI  felt  that  it  was  necessary  to  have  clear 
separation  between  phases  to  allow  for  breakpoints  where  contractors  could  be 
down-selected  between. 

Proposition  3.  This  proposition  was  based  on  the  belief  that  acquisitions 
must  have  clear  performance  targets  and  performance  evaluation  criteria  in 
order  to  be  able  to  use  the  “rolling  down-select  strategy."  The  data  supported 
this  proposition.  All  five  cases  used  clear  performance  targets  and  performance 
evaluation  criteria  and  felt  that  they  were  necessary  elements  for  RDS.  Although 
clear  performance  targets  and  performance  evaluation  criteria  can  be  seen  as 
necessary  elements  for  all  acquisitions,  they  need  to  be  explicitly  clear  for  RDS. 
In  this  strategy,  there  are  multiple  evaluations,  and  sometimes  multiple  teams  of 
evaluators.  Even  slight  ambiguity  in  the  targets  or  evaluation  criteria  could  skew 
the  acquisition  (Fry  ,  1995). 

Proposition  4.  This  proposition  expressed  the  belief  that  acquisitions  had 
to  be  able  to  provide  sufficient  incentive  within  each  phase  to  maintain  multiple 
contractor  efforts.  The  results  of  this  inquiry  was  that  all  cases  felt  that  there 
was  no  need  for  additional  incentive.  However,  these  results  were  based  on  the 
assumption  that  contractors  were  responsible  for  their  own  proposal  and 
associated  costs.  In  reality,  contractors  would  be  incentivized  to  continue  to 
compete  for  award  as  long  as  their  costs  were  being  covered.  This  discovery  led 
the  researcher  to  focus  on  the  question  of  whether  or  not  the  contractor  was 
compensated  for  his  expenses.  This  focus  resulted  in  support  for  this 
proposition.  All  five  cases  felt  that  the  competing  contractors  in  phase  needed  to 
be  compensated  for  their  expenses.  Two  of  the  cases,  EELV  and  EGI  provided 
their  contractors  with  firm  fixed  fee  contracts.  The  other  three  cases  used  cost 
plus  reimbursement  contracts  to  compensate  the  contractors. 
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other  Findings 


During  the  course  of  the  interviews,  case  representatives  being 
interviewed  identified  what  they  felt  were  the  necessary  elements  for  RDS. 

These  suggestions  were  cross-referenced  and  compared  during  further 
interviews  and  analysis. 

Sufficient  Competition  was  seen  as  a  requirement  for  the 
implementation  of  the  “rolling  down-select  strategy.”  This  idea  was  supported 
by  all  cases.  Every  participant  felt  that  it  was  crucial  for  there  to  be  at  least  two 
competing  contractors  for  RDS  to  work.  Although  this  requirement  may  seem 
obvious,  it  is  a  key  consideration.  When  doing  the  initial  acquisition  planning, 
personnel  need  to  make  sure  that  their  market  survey  results  in  at  least  two 
viable  competitors  to  use  RDS. 

Performance  Based  Specifications  were  seen  as  a  necessary  element 
for  the  use  of  the  “rolling  down-select  strategy.”  All  five  cases  felt  that 
performance  based  specifications  were  necessary  for  RDS  to  be  used. 
Performance  based  specifications  are  necessary  because  there  must  be 
differences  in  each  contractor’s  performance  which  can  be  evaluated. 

A  Development  Time  between  18  and  24  months  was  seen  as  a 
necessary  element  to  use  RDS.  Four  of  the  cases  agreed  with  this  idea,  while 
EG  I  did  not  have  the  experience  to  determine  the  proper  development  time.  The 
four  cases  in  agreement  felt  that  this  time  period  made  it  possible  to  perform  the 
source  selection  and  get  the  best  product.  Less  time  might  not  be  long  enough 
while  more  time  would  be  hard  to  manage.  JDAM  felt  that  if  a  program’s 
development  time  was  too  long  the  acquisition  personnel  could  not  effectively 
evaluate  and  manage  the  program.  JASSM  felt  that  if  the  development  time  was 
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shorter  it  would  be  too  difficult  to  effectively  evaluate  and  down-select 
contractors. 

A  Possible  Cost  Savings  was  seen  as  a  necessary  element  by  three  of 
the  five  cases.  JDAM,  JASSM,  and  SBIRS-HI  all  felt  that  a  potential  cost 
savings  was  necessary  to  warrant  the  use  of  RDS.  This  cost  savings  was  seen 
in  the  form  of  a  possible  reduction  in  life-cycle  costs  or  large  unit  buys.  All  three 
of  these  acquisitions  were  realizing  a  cost  savings  and  felt  that  this  savings 
made  the  use  of  the  “rolling  down-select  strategy”  worthwhile. 

A  High  Risk  Acquisition  was  deemed  a  necessary  element  by  all  five 
cases.  All  five  cases  felt  that  RDS  was  an  effective  strategy  for  mitigating  risk 
whether  it  be  technical  risk  or  other  types  of  risk.  A  five  acquisitions  felt  that  a 
high  risk  acquisition  would  be  an  excellent  candidate  for  RDS  because  of  the 
“rolling  down-select”  strategies  ability  to  handle  that  risk.  The  cases  felt  that  the 
breakpoints  between  phases  and  the  review,  feedback,  and  monitoring  of  the 
contractors  made  the  personnel  more  aware  of  the  risks  involved  so  that  they 
were  able  to  better  manage  them. 

Limited  Initial  Funding  was  seen  as  an  element  which  wouid  make  an 
acquisition  a  candidate  for  the  “rolling  down-select  strategy”.  Two  of  the  cases, 
JASSM  and  EELV,  agreed  with  this  statement.  The  rest  of  the  cases  either 
disagreed,  or  did  not  know  if  limited  initial  funding  would  make  a  project  a 
candidate.  The  two  acquisitions  felt  that  the  use  of  RDS  helped  them  overcome 
their  low  funding  and  budget  cuts.  JASSM  and  EELV  pointed  out  that  the 
competition  into  phase  allowed  them  to  set  affordability  as  one  of  their  evaluation 
criteria  and  lower  their  costs. 

An  Acquisition  which  needs  to  Minimize  Schedule  Impact  was  viewed 
as  an  necessary  element  for  RDS.  Four  of  the  five  acquisitions  felt  that  the 
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“rolling  down-select  strategy”  could  be  used  to  minimize  the  schedule  impact  on 
an  acquisition.  EGI  actually  used  RDS  to  minimize  the  schedule  impact  of  a 
protest.  The  other  3  acquisitions  felt  that  RDS  shortened  their  schedule  by 
streamlining  their  efforts.  SBIRS-HI  was  the  only  case  which  felt  that  RDS 
lengthened  their  schedule. 

Conclusion 

This  chapter  consisted  of  the  results  of  the  data  collected  to  analyze  the 
investigate  question  and  related  propositions  to  determine  the  necessary 
elements  required  to  implement  the  “rolling  down-select  strategy.”  The 
conclusions  of  this  research  as  well  as  recommendations  for  future  related 
research  as  this  technique  matures  are  the  subject  of  Chapter  5.  The  research 
conclusions  provide  guidance  to  acquisition  professionals  to  aid  in  their  decision 
of  the  best  acquisition  plan  for  the  program. 
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5.  Conclusion 


The  Department  of  Defense  (DOD)  must  improve  its  weapons  systems 
acquisition.  As  a  part  of  acquisition  reform  under  a  section  of  Lightning  Bolt  10, 
the  “rolling  down-select  strategy”  was  introduced.  RDS  is  a  new  acquisition 
approach  aimed  at  streamlining  the  acquisition  process.  Because  of  the 
newness  of  this  acquisition  approach,  there  are  many  questions  to  be  answered 
on  this  subject.  The  focus  of  this  research  was  to  aid  acquisition  planners  in 
deciding  whether  or  not  they  should  use  this  strategy  based  on  their  particular 
acquisition.  This  exploratory  research  attempted  to  uncover  the  elements 
required  to  implement  the  “rolling  down-select  strategy.” 

The  findings  are  somewhat  varied.  This  variety  was  possibly  due  in  part 
to  the  nature  of  exploratory  research  and  the  relative  newness  and  limited  use  of 
this  technique.  At  the  same  time  that  some  findings  were  varied,  many  trends 
cut  across  the  cases.  This  exploratory  research  revealed  a  set  of  elements 
which  are  necessary  for  the  use  of  the  “rolling  down-select  strategy”.  In  addition, 
further  analysis  of  the  data  indicated  that  there  are  certain  elements  of  a 
program  which  are  not  necessary,  but  lend  themselves  to  the  use  of  RDS.  The 
research  conclusions  are  grouped  into  two  categories:  the  elements  required  for 
the  implementation  of  the  “rolling  down-select  strategy”,  and  elements  of 
programs  which  would  benefit  from  the  use  of  RDS.  This  research  was 
exploratory;  therefore,  findings  and  conclusions  are  preliminary.  This  chapter  will 
conclude  with  ideas  for  future  research  and  some  final  thoughts. 
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Elements  Required  to  Implement  the  “Rolling  Down-Select  Strategy” 

Analysis  done  during  this  exploratory  research  indicated  that  the  following 
elements  were  necessary  elements  to  successfully  implement  the  “rolling  down- 
select  strategy.”  Table  6  includes  a  listing  of  these  elements. 

State  of  the  Art  or  State  of  the  Practice  technology  is  a  necessary 
element  to  implement  RDS.  The  technology  must  be  either  state  of  the  art  or 
state  of  the  practice  to  use  the  “rolling  down-select  strategy.”  The  acquisition 
must  be  of  a  sufficient  technological  state  to  warrant  the  use  of  the  “rolling  down- 
select  strategy.”  At  the  same  time,  the  acquisition  must  not  be  of  such  a  high 
technological  state  that  it  is  not  possible  to  measure  and  evaluate  progress.  All 
cases  studied  agreed  that  an  acquisition’s  technology  must  be  either  state  of  the 
art  or  state  of  the  practice  to  use  RDS. 

There  must  be  a  clear  separation  between  phases  to  implement  the 
“rolling  down-select  strategy.”  Acquisitions  must  be  structured  to  be  broken  up 
into  distinct  phases  for  RDS.  There  must  be  breakpoints  between  phases  where 
contractors  can  be  down-selected  to  the  next  phase.  This  clear  separation  is 
also  necessary  so  that  the  contractors  can  be  evaluated  and  maintained  during 
manageable  fragments  of  time.  All  of  the  cases  felt  that  a  clear  separation 
between  phases  was  a  necessary  element  for  the  “rolling  down-select  strategy.” 

There  must  be  clear  performance  targets  and  performance 
evaluation  criteria  for  RDS  to  be  successful.  Acquisitions  must  lend 
themselves  to  selecting  clear  targets  and  evaluation  criteria  for  making  down- 
select  decisions.  The  targets  and  evaluation  criteria  must  be  clear  and  known  to 
both  the  competing  contractors  as  well  as  the  Government  evaluators.  Because 
of  the  fact  that  multiple  contractors  are  being  evaluated  at  the  same  time  there 
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must  be  clear  targets  and  evaluation  criteria  which  can  be  used  to  fairly  evaluate 
all  sources.  Without  clear  targets  and  evaluation  criteria  provided  to  each 
contractor,  the  Government  becomes  susceptible  to  possible  protests  and  a 
flawed  acquisition  process.  All  cases  interviewed  agreed  that  clear  performance 
targets  and  clear  performance  evaluation  criteria  were  necessary  for  the  “roiling 
down-select  strategy”  to  be  successful. 

There  must  be  sufficient  funding  to  compensate  the  contractors  for 
their  efforts  for  RDS  to  work.  The  acquisition  must  have  enough  funding  to 
compensate  multiple  contractors  for  their  work  done  during  the  phase.  For  this 
to  be  accomplished,  Government  cost  estimators  need  to  calculate  the  total 
costs  involved  in  the  acquisition  and  estimate  the  costs  required  for  each 
contractor.  If  their  are  insufficient  funds  to  down-select  more  than  one 
contractor,  this  strategy  is  not  practical.  In  all  of  the  acquisitions  studied,  the 
contractors  were  compensated  through  either  fixed  price,  or  cost  plus  contracts. 
All  of  the  acquisitions  felt  it  was  necessary  to  have  sufficient  funds  to 
compensate  the  competing  contractors  for  their  work  in  phase. 

Adequate  competition  must  exist  for  the  “rolling  down-select  strategy” 
to  be  used.  At  least  two  qualified  sources  must  exist  for  the  “rolling-down  select 
strategy”  to  be  used.  Although  it  may  seem  obvious  that  sufficient  competition  is 
a  necessary  element  for  RDS,  it  should  not  be  overlooked.  As  a  checklist  for 
acquisition  planners  to  use  when  deciding  whether  or  not  to  implement  RDS,  the 
need  for  adequate  competition  may  be  a  box  which  can  be  checked  easily,  but  it 
still  must  be  checked  for  RDS  to  work.  When  doing  the  market  survey  for  an 
acquisition,  acquisition  planners  need  to  ensure  that  there  are  two  viable  sources 
who  can  compete  in  phase  for  this  acquisition.  All  cases  made  it  a  point  to 
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mention  that  adequate  competition  was  a  requirement  for  the  “rolling  down-select 
strategy”  to  be  used. 

The  specifications  must  be  performance  based  to  make  RDS  work. 
Performance  based  specifications  give  the  contractor  discretion  on  how  to  fulfill 
the  terms  of  the  contract.  Performance  based  specifications  are  necessary  to 
make  it  possible  for  the  contractor’s  performance  to  be  distinct  enough  to  be 
evaluated  by  the  Government  selectors.  Design  based  specifications  provided 
by  the  Government  lend  themselves  to  a  single  contractor  award  because  the 
Government  can  be  held  liable  for  defective  specifications  by  the  contractor 
using  them.  All  cases  researched  mentioned  that  performance  based 
specifications  were  necessary  for  RDS  to  work. 


Table  6.  Required  Elements 


1 .  State  of  the  Art  or  State  of  the  Practice 

2.  There  must  be  a  clear  separation  between  phases 

3.  There  must  be  clear  targets  and  evaluation  criteria 

4.  There  must  be  sufficient  funding  to  compensate  the  contractors 

5.  Adequate  competition  must  exist 

6.  The  specifications  must  be  performance  based 
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Programs  Which  would  Benefit  from  RDS 


Table  7  includes  a  list  of  the  programs  which  would  benefit  from  the  use  of 

RDS. 

Programs  with  a  development  time  between  18  and  24  months  could 
benefit  from  the  use  of  RDS.  Programs  with  a  development  time  less  than  18 
months  may  not  lend  themselves  to  evaluating  multiple  contractors  fairly  and 
effectively.  Programs  longer  than  24  month  development  time  might  make 
provide  too  much  excess  information  and  investment  and  make  it  more  difficult  to 
make  evaluations  and  down-selections.  Four  of  the  five  cases  felt  that  a 
program  with  a  development  time  between  18  and  24  months  would  benefit  from 
the  use  of  the  “rolling  down-select  strategy.” 

Programs  with  a  possible  cost  savings  could  benefit  from  the  use  of 
RDS.  Programs  with  long  life  cycles  or  large  batch  costs  could  benefit  from  the 
use  of  the  “rolling  down-select  strategy.”  Because  of  the  evaluations  and 
feedback  in  phase  opportunities  for  cost  savings  can  be  taken  advantage  of  as 
they  are  seen.  In  addition,  the  fact  that  affordability  is  often  an  evaluation 
characteristic  during  in-phase  competition,  sources  are  motivated  to  cut  costs 
and  look  for  new  ways  to  provide  a  more  affordable  product  to  the  Government 
in  order  to  be  down-selected  to  the  next  phase.  Three  of  the  cases  analyzed  felt 
that  programs  with  a  possible  cost  savings  could  benefit  from  RDS. 

Programs  with  High  Risk  could  benefit  from  RDS.  The  “rolling  down- 
select  strategy”  is  able  to  mitigate  many  types  of  risk.  The  breakpoints  between 
phases  and  the  continued  competition  into  the  phases  allows  the  Government  to 
mitigate  the  risk  of  program  failure.  The  review,  feedback  and  monitoring  of  the 
contractor  also  makes  it  possible  to  manage  risk.  Programs  which  contain  high 
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risk  areas  could  benefit  from  the  “rolling  down-select  strategies”  mitigation  of  this 
risk.  All  cases  felt  that  high  risk  acquisitions  could  benefit  from  RDS. 

Programs  with  Limited  Initial  Funding  could  survive  if  the  “rolling  down- 
select  strategy”  is  used.  Programs  with  minimal  or  insufficient  funding  could 
possibly  remain  viable  by  using  RDS.  The  phased  approach  of  the  “rolling  down- 
select”  strategy  provides  a  time  period  from  entry  into  one  phase  until  the 
breakpoint  before  the  next  phase  where  acquisition  personnel  can  apply  for 
more  funding  based  on  the  programs  staged  success.  Two  of  the  cases  were  hit 
by  budget  cuts  and  RDS  made  it  possible  for  them  to  take  their  program  through 
a  program  definition-risk  reduction  phase  while  they  were  waiting  for  more  funds. 
At  the  same  time,  the  affordability  evaluation  enhanced  by  the  competition  in 
phase  made  it  possible  for  the  programs  to  lower  their  costs  and  meet  their 
funding  levels. 

Programs  attempting  to  minimize  schedule  impact  could  be 
successful  by  using  the  “rolling  down-select  strategy.”  The  “rolling  down-select 
strategy”  provides  a  potential  for  time  savings  through  the  rolling  evaluation 
process.  By  the  time  the  phase  is  complete,  the  decision  of  who  will  be  down- 
selected  to  the  next  phase  is  either  already  complete,  or  almost  complete.  This 
process  makes  it  possible  to  take  a  program  from  one  phase  to  the  next  without 
a  long  period  in  between  phases  to  evaluate  and  down-select  sources.  In 
addition,  in  acquisitions  where  there  has  been  a  protest,  it  is  possible  to  institute 
RDS  to  move  forward  with  the  acquisition  without  redoing  the  source  selection  as 
EGI  did.  Four  of  the  cases  felt  that  an  acquisition  which  needed  to  minimize 
schedule  impact  could  benefit  from  the  use  of  RDS. 


57 


Table  7.  Programs  Which  Would  Benefit 


1.  A  Development  Time  Between  18  and  24  Months 

2.  A  Possible  Cost  Savings 

3.  High  Risk 

4.  Limited  Initial  Funding 

5.  Attempting  to  Minimize  Schedule  Impact 


All  of  these  conclusions  will  help  acquisition  planners  to  better  implement 
the  "rolling  down-select  strategy."  The  purpose  of  the  Lightning  Bolt  10 
acquisition  reform  initiative,  which  includes  RDS,  is  to  reduce  life  cycle  time  and 
streamline  the  acquisition  process.  By  implementing  this  technique  when  these 
elements  are  present,  acquisition  professionals  ensure  the  best  opportunity  for 
the  successful  implementation  of  this  acquisition  reform  technique,  the  "rolling 
down-select  strategy."  Successful  implementation  may  mean  a  streamlined 
acquisition  process  and  reduced  life  cycle  time. 

Recommendations  for  Future  Research 

One  of  the  goals  of  this  exploratory  research  was  to  shed  light  on  the 
“rolling  down-select  strategy”  and  lay  a  foundation  for  further  research  .  The 
following  are  some  suggested  related  research  topics.  Some  ideas  are  to 
validate  and  expand  the  preliminary  findings  of  this  study.  Other  possible 
research  topics  delve  into  related  aspects.  The  “rolling  down-select  strategy” 
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was  a  fairly  new  derivation  of  traditional  acquisition  techniques.  This  study 
should  be  conducted  again  after  this  technique  has  been  in  existence  for  several 
years. 

Research  Validation.  This  research  discovered  the  elements  required  for 
the  “rolling  down-select  strategy”  to  be  successful.  The  support  for  these 
elements  was  preliminary.  Further  research  should  be  conducted  to  validate 
these  findings.  This  research  explored  the  “rolling  down-select  strategy” 
acquisitions  used  by  AFMC  only.  Future  research  could  expand  the  study  to 
other  major  commands  in  the  Air  Force,  or  other  branches  of  the  Department  of 
Defense.  A  wise  candidate  would  be  a  study  and  comparison  of  the  National 
Aeronautical  and  Space  Administration’s  use  of  RDS  compared  with  the  Air 
Force’s. 

Source  Selection  Sensitive.  One  of  the  aspects  of  the  “rolling  down-select 
strategy”  a  single  source  is  not  awarded  a  contract  until  the  final  phase.  During 
each  phase  sources  are  still  being  evaluated  for  selection.  Because  of  this 
continuous  source  selection,  ail  each  contractor’s  information  is  source  selection 
sensitive  until  final  award.  Research  could  be  conducted  into  whether  or  not  a 
policy  needs  to  be  developed  for  this  source  selection  sensitivity  and  any  other 
requirements  of  RDS. 

Quantitative  Analysis.  At  the  time  this  research  was  conducted,  there 
were  not  enough  acquisitions  which  used  the  “rolling  down-select  strategy”  to 
quantifiably  evaluate  this  thesis.  When  there  are  a  sufficient  number  of 
acquisitions  which  have  used  RDS,  a  sample  should  be  tested  statistically.  A 
quantitative  analysis  could  be  done  to  determine  exactly  how  important  each  of 
the  required  elements  are  to  RDS.  A  statistical  analysis  could  also  be  done  to 
compare  and  contrast  the  traditional  acquisition  approach  to  RDS  based  on  cost 
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and  schedule  overruns  once  a  sufficient  number  of  RDS  acquisitions  are 
completed. 

Advantages/Disadvantages.  At  the  time  this  exploratory  research  was 
conducted  interviewees  were  unable  to  clarify  the  advantages  or  disadvantages 
of  the  “rolling  down-select  strategy”  versus  a  traditional  acquisition  approach. 

The  response  to  questions  of  a  cost  or  time  savings  was  that  the  acquisition 
personnel  were  unable  to  quantify  if  there  had  been  a  cost  or  time  savings  or 
other  advantages  or  disadvantages  at  the  program’s  stage.  In  the  future,  after 
these  acquisitions  are  completed  and  more  acquisitions  use  RDS  this  topic  could 
be  researched.  There  were  a  few  possible  advantages  discovered  during  this 
research.  Possible  advantages  are  cost  savings,  time  savings,  reduced  risk  of 
awarding  a  contract  where  a  contractor  was  unable  to  perform,  more  feasible 
options  to  meet  system  requirements,  increased  performance,  and  a  better 
product.  Some  possible  disadvantages  are  cost  increases  because  of  more 
contractors  to  evaluate  and  visit,  schedule  delays,  and  more  required  man-hours. 
When  questioned  about  the  necessary  elements  such  as  more  manpower  to 
work  a  SPO,  ail  of  the  cases  said  that  they  did  not  have  to  hire  any  more  workers 
but  all  of  the  personnel  had  to  work  longer  hours.  Research  into  the  advantages 
and  disadvantages  of  the  “rolling  down-select”  strategy  should  be  taken  up  at  a 
later  date  when  more  acquisitions  are  complete  and  acquisition  personnel  can 
determine  whether  or  not  they  had  savings  or  losses. 

Education  and  Training.  Research  should  be  done  to  look  into  how  well 
acquisition  personnel  are  trained  and  exposed  to  new  acquisition  techniques. 
AFMC  personnel  indicated  in  interviews  that  they  had  been  given  little  if  any 
exposure  to  this  acquisition  technique.  AFIT’s  PCE  program  should  be 
researched  to  determine  if  acquisition  approach  alternatives  are  being  taught  or 
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explored.  Research  should  be  conducted  to  determine  whether  or  not  the 
continuing  education  courses  are  preparing  acquisition  personnel  to  devise  or 
use  new  acquisition  techniques  when  necessary. 

Develop  a  New  RDS  Technique  Based  on  Qualities  of  Current  Variations^ 
This  research  discovered  that  variations  of  the  “rolling  down-select  strategy” 
strategy  are  currently  being  used.  Research  could  be  conducted  to  analyze 
each  variation  and  determine  what  mix  of  these  variations  would  provide  the 
optimal  benefits  of  the  strategy.  Research  could  be  conducted  to  develop  an 
entirely  new  acquisition  strategy  based  on  RDS  and  traditional  acquisition 
approaches.  Research  could  also  be  conducted  to  determine  whether  or  not  the 
entire  acquisition  process  should  be  changed. 

Final  Thoughts 

In  today’s  post  cold  war  era,  the  Department  of  Defense  (DOD)  must  take 
action  in  the  face  of  massive  downsizing  and  huge  budget  cuts.  The  Department 
of  Defense  is  attempting  to  take  action  through  action  through  acquisition  reform 
measures  such  as  the  “rolling  down-select  strategy.”  RDS  is  a  new  acquisition 
approach  aimed  at  streamlining  the  acquisition  process.  Because  of  the 
newness  of  this  acquisition  approach,  there  are  many  questions  to  be  answered 
on  this  subject.  The  focus  of  this  research  was  to  aid  acquisition  planners  in 
deciding  whether  or  not  they  should  use  this  strategy  based  on  their  particular 
acquisition.  This  exploratory  research  uncovered  the  elements  required  to 
implement  the  “rolling  down-select  strategy.” 

This  exploratory  research  revealed  that  there  are  a  set  of  elements  which 
are  necessary  for  the  use  of  the  “rolling  down-select  strategy”.  In  addition. 
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further  analysis  of  the  data  indicated  that  there  are  certain  elements  of  a 
program  which  are  not  necessary,  but  lend  themselves  to  the  use  of  RDS. 
These  elements  can  now  be  used  by  acquisition  planners  to  decide  whether  or 
not  their  acquisition  should  use  the  “rolling  down-select  strategy.” 


Appendix  A:  Case  Study  Pattern  Codes 


Table  8.  Pattern  Codes 


Pattern  Code 

Operational  Definition 

State-Simple 

■ 

State  of  the  Simple;  The  date  indicates  that  the  technology  in  the  system  being 
acquired  was  simple. 

State-Practice 

■ 

State  of  the  Practice:  The  data  indicates  that  the  technology  in  the  system  being 
acquired  was  state  of  the  practice. 

State-Art 

State  of  the  Art:  The  data  indicates  that  the  technology  in  the  system  being 
acquired  was  state  of  the  art. 

State-  Exp 

State  of  the  Experimental:  The  data  indicates  that  the  technology  in  the  system 
being  acquired  was  experimental. 

Tech-Lev-nec 

Technology  level  necessary:  The  data  indicates  that  the  level  of  technology  of  the 
system  played  a  big  role  in  the  decision  to  use  RDS. 

EZ-Sep 

Easy  to  separate:  The  data  indicates  that  the  program  was  easy  to  separate  into 
phases. 

EZ-Sep-Nec 

Easy  separation  was  necessary;  The  data  indicates  that  program  personnel  felt  it 
was  necessary  for  the  program  to  be  easily  separated  into  phases. 

CIr-Targ 

Clear  Targets:  The  data  Indicates  that  there  were  clear  performance  targets  and 
criteria  which  were  used  in  making  down-selections 

CIr-Targ-Nec 

Clear  Targets  Necessary:  The  data  indicates  that  clear  performance  targets  were 
necessary  for  RDS 

Ktor-Inc 

Contractor  Incentive;  The  data  indicates  that  the  contractors  were  given  extra 
incentives  to  compete  for  successive  down-selection 

Ktor-Inc-Nec 

Contractor  Incentive  Necessary:  The  data  indicates  that  program  personnel  felt 
that  it  was  necessary  for  contractors  to  be  given  extra  incentive  to  continue  to 
compete. 

Ktor-comp-CP 

Contractor  Compensation  Cost  Reimbursement:  The  data  indicates  that  the 
contractors  were  compensated  for  all  their  costs  through  a  cost  reimbursement 
contract. 

Ktor-comp-FF 

Contractor  Compensation  Fixed  Price:  The  data  indicates  that  the  contractors 
were  compensated  through  a  fixed  priced  contract. 

Ktor-Comp-Nec 

Contractor  Compensation  Necessary:  The  data  indicates  that  program  personnel 
felt  that  It  was  necessary  for  the  contractors  to  be  compensated  for  their  work 
during  the  phase. 

Ad-Comp-Nec 

Adequate  Competition  Necessary:  The  data  indicates  that  the  program  personnel 
felt  that  adequate  competition  (at  least  two  sources)  was  a  necessary  element  for 
RDS 

Perf-Bse-Spec 

Performance  Based  Specifications:  The  data  indicates  that  the  program 
personnel  felt  that  performance  based  specifications  were  necessary  for  RDS. 
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Table  8  (Continued) 


Pattern  Codes 

Operational  Definitions 

Dev-Time  >18  <24 

Development  Time;  The  data  indicates  that  a  planned  development  time 
of  18-24  months  was  necessary  for  RDS 

Poss-Cost-Sav 

Possible  Cost  Savings:  The  data  indicates  that  program  personnel  felt 
that  it  was  necessary  for  their  to  be  a  possible  cost  savings  through  lower 
life  cycle  costs,  or  large  production  numbers  to  benefit  from  RDS. 

Hi-Risk 

High  Risk:  The  data  indicates  that  program  personnel  felt  that  it  was 
necessary  for  a  program  to  have  a  lot  of  risk  which  needed  to  be  mitigated 
to  benefit  from  RDS. 

Lo-Fund 

Low  Funding:  The  data  indicates  that  program  personnel  felt  that  a 
program  with  minimal  to  adequate  funding  would  benefit  from  RDS. 

Min-Sched-Imp 

Minimize  Schedule  Impact:  The  data  indicates  that  the  program 
personnel  felt  that  a  program  where  there  was  an  upheld  protest  would 
benefit  from  RDS  by  minimizing  the  schedule  impact. 
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Appendix  B:  Interview  Questions 


The  following  questions  were  used  to  structure  the  data  collection 

process. 

1 .  What  program  were  you  involved  in? 

2.  Did  this  program  use  the  “rolling  down-select  strategy?” 

3.  Had  you  ever  previously  been  exposed  to  this  strategy? 

4.  How  did  you  learn  about  the  existence  “rolling  down-select  strategy?” 

5.  Where  did  you  find  guidance  to  implement  the  “rolling  down-select  strategy?” 

6.  Which  document  did  you  find  to  be  most  helpful  in  terms  of  guidance? 

7.  Whose  decision  was  it  to  use  the  “rolling  down-select  strategy?” 

8.  In  developing  the  acquisition  development  timeline  for  the  program  was  it 
decided  at  the  outset  that  the  traditional  acquisition  approach  or  some  other 
method  of  source  selection  besides  the  “rolling  down-select”  would  be 
applied? 

IF  YES; 

9.  What  was  the  original  source  selection  procedure  that  you  considered  if  the 
“rolling  down-select  strategy”  was  not  the  first  choice? 

IF  NO 

9.  Why  was  no  consideration  given  to  another  method? 

10.  What  were  the  characteristics  or  features  of  the  system  being  acquired  that 
led  to  a  decision  to  use  the  “rolling  down-select  strategy?” 

1 1 .  Reflecting  on  how  this  program  went,  or  compared  to  other  programs,  was 

the  “rolling  down-select  strategy”  a  wise  choice  for  this  acquisition? 

12.  Why? 
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1 3.  Does  the  anticipated  length  of  the  development  period  influence  the  decision 
to  use  the  “rolling  down-select  strategy?” 

IF  YES: 

14.  Which  of  the  following  projected  completion  times  (compared  to  completion 
times  for  other  acquisitions)  appear  to  be  applicable  to  the  “rolling  down- 
select”  strategy?” 

Short  Medium  Long  Extremely  Long 

15.  What  stage/phase  is  this  program  currently  at? 

16.  How  long  has  this  program  been  in  existence 

17.  How  long  did  this  source  selection  take  to  complete,  or  if  it  is  not  yet 
complete,  how  long  will  it  take? 

18.  How  would  you  describe  the  level  of  funding  for  this  program  compared  to 
other  programs? 

Minimally  Funded  Adequately  Funded  Well-Funded  Extremely  Well-Funded 

19.  What  total  dollar  funding  amount  did  you  base  question  18’s  evaluation  on? 

20.  In  developing  the  budget  did  you  have  to  make  special  allowances  for  the 
costs  imposed  by  the  “rolling-down-select  strategy?” 

21 .  What  additional  costs  seemed  to  be  brought  on  by  this  process  compared  to 
a  single  award  source  selection? 

22.  What  was  the  level  of  technology  involved  in  this  program? 

Simple  State  of  the  practice  State  of  the  art  Experimental 

23.  Did  the  level  of  technology  being  incorporated  into  the  weapons  system  play 
a  big  role  in  the  decision  to  use  the  “rolling  down-select  strategy? 

24.  Did  the  level  of  technology  employed  in  the  production  processes  play  a  big 
role  in  the  decision  to  use  the  “rolling  down-select  strategy?” 

25.  Was  a  clear  separation  between  phases  necessary  to  use  the  “rolling  down- 
select  strategy?” 

26.  Was  this  acquisition  easy  to  separate  into  phases  for  the  use  of  the  “rolling 
down-select  strategy?” 
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27.  What  was  the  nature  of  the  performance  targets  and  criteria  which  were  used 
in  deciding  which  contractors  would  be  down-selected  to  the  next  phase? 

28.  Were  there  clear  performance  targets  and  criteria  to  use  in  deciding  which 
contractors  would  be  down-selected  to  the  next  phase? 

29.  Were  contractors  given  special  incentives  to  continue  to  compete  to  be  down- 
selected  to  successive  phases? 

30.  What  were  the  contractors  compensated  for?,  eg  proposals,  all  costs. . . 

31 .  What  was  the  form/type  of  contract  were  signed  with  the  contractors? 

32.  What  are  the  necessary  elements  a  program  needs  to  use  the  “rolling  down- 
select  strategy?” 

33.  What  characteristics  of  a  program  would  lead  you  to  think  that  it  may  be  a 
candidate  for  the  “rolling  down-select  strategy? 

34.  Would  the  presence  of  one  of  these  characteristics  be  sufficient  or  is  there 
some  minimum  set  of  characteristics  that  is  required? 

35.  Are  there  any  elements  that  this  program  did  not  have  that  could  have  been 
beneficial  to  the  “rolling  down-select  strategy?” 

36.  If  you  had  it  to  do  over  again,  would  you  select  the  “rolling  down-select 
strategy”  on  this  acquisition? 

37.  Why? 

38.  Should  the  Air  Force  ever  use  the  “rolling  down-select  strategy”  in  the 

future? 

39.  If  so,  on  what  type  of  program? 

40.  What  were  the  advantages  of  using  the  “rolling  down-select  strategy”  as 
opposed  to  another  source  selection  technique? 

41.  Was  there  a  cost  savings,  if  so  how  much,  and  what  caused  it 

42.  Was  there  a  time  savings,  if  so  how  much,  and  what  caused  it? 
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43.  Do  you  feel  that  there  was  a  reduced  risk  of  awarding  a  final  contract  where 
the  contractor  was  unable  to  perform? 

44.  Did  the  “rolling  down-select  strategy  provide  you  with  more  feasible  options 
to  meet  the  system  requirements  and  reduce  the  risk  involved  with  a  single 
contractor  single  solution  acquisition? 

45.  Did  the  continuing  of  competition  into  the  program  phase  increase 
performance? 

46.  Did  the  “rolling  down-select  strategy”  make  it  easier  to  take  advantage  of 
technological  breakthroughs  because  of  separate  contracts  for  separate 
phases? 

47.  What  were  the  disadvantages  of  using  the  “rolling  down-select  strategy”  as 
opposed  to  another  source  selection  technique? 

48.  Was  there  a  significant  increased  cost  associated  with  the  “rolling  down- 
select  strategy,  if  so,  how  much,  and  what  caused  it? 

49.  Did  the  use  of  the  “rolling  down-select  strategy”  require  a  larger  SPO  or  more 
manpower? 

50.  Are  you  aware  of  any  other  programs  that  have  employed  the  “Rolling 
Down-Select  Strategy?” 

51.  Do  you  have  any  other  comments  or  suggestions  concerning  the  “rolling 
down-select  strategy?” 
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